-- Logs begin at Mon 2022-05-16 17:56:38 UTC, end at Mon 2022-05-16 19:44:04 UTC. -- May 16 18:36:28.612850 dv-cinder-841125-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 16 18:36:28.620506 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 16 18:36:28.624757 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 16 18:36:28 2022] *** May 16 18:36:28.624757 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 16 18:36:28.624757 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 16 18:36:28.624757 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: nodename: dv-cinder-841125-5-iscsi May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: machine: x86_64 May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: clock source: unix May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: pcre jit disabled May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: detected number of CPU cores: 4 May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: current working directory: / May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: detected binary path: /usr/bin/uwsgi-core May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: your processes number limit is 47680 May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: your memory page size is 4096 bytes May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: detected max file descriptor number: 2048 May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: lock engine: pthread robust mutexes May 16 18:36:28.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: thunder lock: enabled May 16 18:36:28.625580 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 16 18:36:28.625580 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 16 18:36:28.672232 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: Python main interpreter initialized at 0x5580bf6e49c0 May 16 18:36:28.672232 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: python threads support enabled May 16 18:36:28.672232 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: your server socket listen backlog is limited to 100 connections May 16 18:36:28.672232 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: your mercy for graceful operations on workers is 90 seconds May 16 18:36:28.672657 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: mapped 403077 bytes (393 KB) for 2 cores May 16 18:36:28.672731 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: *** Operational MODE: preforking *** May 16 18:36:28.673029 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: *** uWSGI is running in multiple interpreter mode *** May 16 18:36:28.673029 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: spawned uWSGI master process (pid: 91529) May 16 18:36:28.673240 dv-cinder-841125-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 16 18:36:28.674214 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: spawned uWSGI worker 1 (pid: 91542, cores: 1) May 16 18:36:28.674214 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: spawned uWSGI worker 2 (pid: 91543, cores: 1) May 16 18:36:28.674360 dv-cinder-841125-5-iscsi devstack@keystone.service[91529]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 16 18:36:34.374094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 18:36:34.374094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: from cryptography.utils import int_from_bytes May 16 18:36:34.374910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 18:36:34.374910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: from cryptography.utils import int_from_bytes May 16 18:36:34.375520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 18:36:34.375520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: from cryptography.utils import int_from_bytes May 16 18:36:34.375660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 16 18:36:34.375660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: from cryptography.utils import int_from_bytes May 16 18:36:34.425384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 16 18:36:34.425654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 16 18:36:34.425752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] command line args: [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 16 18:36:34.426021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 16 18:36:34.426196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ================================================================================ {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 16 18:36:34.426447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] admin_token = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.426691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] config_dir = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.426877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.427176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] config_source = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.427269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] debug = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.427415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.428914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.429129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.429378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.429635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.429873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.430071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_config_append = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.430226 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.430400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_dir = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.430573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_file = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.430811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.431060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.431305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.431550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.431722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.431897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.432136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.432378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.432480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 16 18:36:34.432692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 16 18:36:34.432791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.432865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] command line args: [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 16 18:36:34.432934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.433135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 16 18:36:34.433233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.433296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ================================================================================ {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 16 18:36:34.433353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.433531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] admin_token = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.433736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.433926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] config_dir = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.434037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.434348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.434543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.434654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] config_source = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.434654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] debug = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.434654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] publish_errors = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.435951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_config_append = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.436999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_dir = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.437238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] standard_threads = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.437369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_file = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.437495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.437620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.437769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.437908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] use_journal = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] use_json = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] use_stderr = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.438972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] use_syslog = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.439098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.439236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.439370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.439543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.439685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.439814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.440003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.440141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.440275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.440275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.440520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.440676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.440879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.441019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.441146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.441283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.441411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.441555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.441693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.441693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.441693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.442055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.442055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] publish_errors = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.442309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.442437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.442689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.442814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.442947 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.443074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.443216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.443349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.443477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.443603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.443741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] standard_threads = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.443868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.443994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.443994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.444238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.444440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.444576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.444743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] use_journal = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.444881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.445009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] use_json = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.445009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] use_stderr = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.445270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.445404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] use_syslog = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.445532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.445664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 16 18:36:34.445791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.445918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.446963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.447868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.448968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.449869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.450988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.451905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.452885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.453908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.454152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.455101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.455275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.455468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.455662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.455891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.456058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.456256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.456448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.456702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.456868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.457101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.457267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.457458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.457663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.457854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.458045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.458272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.458480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.458705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.458908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.459106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.459274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.459500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.459665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.459895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.460061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.460237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.460399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.460540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.460723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.460862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.461733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.462951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.463238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.463391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.463536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.463667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.463795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.463927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.464987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465635 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.465974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.466960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.467884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.468011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.468138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.468265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.468913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.469957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.470883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.external = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.471927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.password = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] auth.token = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.472861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.473942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.474881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.475895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.476932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.external = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.477981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.password = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] auth.token = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.478952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.479876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.480936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.481997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.482648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483222 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.483879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.484007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.484765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.484902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485031 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.485973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.486897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.487956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.488924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.489874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490261 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.490924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491315 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.491975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.492990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493261 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.493912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.494957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.495874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.496955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.497094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.497224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.497353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.497481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.497627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.497845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.498939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.499884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.500984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.501910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.502968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.503890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.504985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.505909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.506965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.507888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.508876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] role.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.509996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] role.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.510916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.511968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.512902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.513955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.514874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] role.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515005 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] role.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.515949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.516922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.517975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.518997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.519905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.520982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.521123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.521253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.521393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.521579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.521716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.521852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.522948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.523878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.connection = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.524862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.525924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.526968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527226 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.527890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.528954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.connection = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.529211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.530371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.532281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 16 18:36:34.532281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91542) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 16 18:36:34.542872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.543101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.543326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.546232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.546464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.547231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.552744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.555868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.581041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.581287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.582781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.583013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.635080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.635320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.687073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.687343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.687919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.688150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.753217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.753478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.753632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.753793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.753939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.754950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.777908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.778130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.778407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.778577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.778720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.778859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.778999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.779958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.791456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.791712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.806518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.806854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.807985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.808119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.808253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.808396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91543) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.808543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91542) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 16 18:36:34.825429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.825706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 18:36:34.825844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.826008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.826145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 18:36:34.826279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 18:36:34.826413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.826546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.826678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 16 18:36:34.826820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.826968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.827102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.827236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.827369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.827510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.827510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.838581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.838856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.839123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.839402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.843139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.843465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.844232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.844512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.848013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.851305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.851584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.851862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.852250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.852528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.852695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.852831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.853063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.853330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.853597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.853865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.854126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.855818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.856096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.856371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.856781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.857068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.857329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.857602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.857876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.858145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.858416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.858681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.880540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.882181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.882456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.885234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.886816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.887094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.887748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.888025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.888300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.892932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.893210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.893486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.896116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.896394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.900792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.900995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.901081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.901282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.901562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.901842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.902118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.902390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.902658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.902926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.908035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.908408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.922228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.922509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.926837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.927002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.927103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.930587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.930880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.931575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.935190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.935483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.935789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.939382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.940456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.943087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.943382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.943661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.943937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.944020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.944210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.944483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.947720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.948022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.948305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.948586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.948893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.949172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.963838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.964218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.964484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.964772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.969312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.969691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.969959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.970218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.973165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.974880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.975152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.978060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.978234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.978461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.978747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.979034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.979309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.979775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.980059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.983117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.983399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.983677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.983962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.984263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.992619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.992922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.993201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.993489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.993768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.994056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.994331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.997574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:34.997855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.998134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.998422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.998699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.998988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:34.999269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.012664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.016462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.018216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.019775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.021397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.021559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.023098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.023378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.023665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.024890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.026484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.028092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.028370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.028677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.031059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.032950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.036161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.036364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.036451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.036737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.037016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.037967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.041203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.041488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.041765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.042035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.046162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.046460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.046764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.051167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.051463 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.051751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.054551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91543) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.054830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.055104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.055377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.055648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.055931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.056202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.056476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.056771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.057054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.057321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.057596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.057857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.058116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91543) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.059634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91542) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 16 18:36:35.059914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.060190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.060464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.060760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.061049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.061319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.061592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.061857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.062134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.063004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.063202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.063444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.063702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91542) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 16 18:36:35.097419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5580bf6e49c0 pid: 91543 (default app) May 16 18:36:35.100772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54f67-75fe-4d4f-9fd9-98d94ae3b355 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:35.101069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54f67-75fe-4d4f-9fd9-98d94ae3b355 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:35.101353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54f67-75fe-4d4f-9fd9-98d94ae3b355 None None] PATH_INFO: `/v3` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:35.102654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5580bf6e49c0 pid: 91542 (default app) May 16 18:36:35.105516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Mon May 16 18:36:35 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:43.122103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc18dc-f1bd-4b0e-b2e2-a689f6d83ee1 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:43.122685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc18dc-f1bd-4b0e-b2e2-a689f6d83ee1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:43.122685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc18dc-f1bd-4b0e-b2e2-a689f6d83ee1 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:43.127586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:43.132928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:43.133341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:43.133510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:43.163460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG oslo_db.sqlalchemy.engines [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 16 18:36:43.344607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG passlib.handlers.bcrypt [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91543) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 16 18:36:43.345052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG passlib.handlers.bcrypt [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91543) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 16 18:36:43.401415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:43.403029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:43.529306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 16 18:36:43.678115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc18dc-f1bd-4b0e-b2e2-a689f6d83ee1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:43.711938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG oslo_db.sqlalchemy.engines [None req-dedc18dc-f1bd-4b0e-b2e2-a689f6d83ee1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91542) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 16 18:36:43.836879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc18dc-f1bd-4b0e-b2e2-a689f6d83ee1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:43.841074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:43.841727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:43.842010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:43.842287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:43.850740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:43.901084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING keystone.common.rbac_enforcer.enforcer [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 16 18:36:43.901424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b1343-d589-48c4-907a-77170a36ab89 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:43.912005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Mon May 16 18:36:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:36:43.918163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:43.932508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae692f-90f9-4cde-b610-75491f57a7cb None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:43.936869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:43.937966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:43.938246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:43.938513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] PATH_INFO: `/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:43.939979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] RBAC: Authorizing `identity:get_project(project_id=b27a519f15bc4f1e98fc811594ab8cdd)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:43.990176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING keystone.common.rbac_enforcer.enforcer [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 16 18:36:43.990510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f59c72-c666-4722-807c-a84b0401b5ca admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:43.995069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Mon May 16 18:36:43 2022] GET /identity/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:46.177796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e6890-efa2-4e5f-9593-2f6a5b49c7c6 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:46.178322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e6890-efa2-4e5f-9593-2f6a5b49c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:46.178322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e6890-efa2-4e5f-9593-2f6a5b49c7c6 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:46.182378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:36:46.188254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:46.188687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:46.188839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:46.208406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:46.210028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:46.217803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 16 18:36:46.239769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e6890-efa2-4e5f-9593-2f6a5b49c7c6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:46.251908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e6890-efa2-4e5f-9593-2f6a5b49c7c6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:46.255976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb124c-a64b-42f3-9f56-179e26363f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:46.256584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb124c-a64b-42f3-9f56-179e26363f27 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:46.256890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb124c-a64b-42f3-9f56-179e26363f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:46.257160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb124c-a64b-42f3-9f56-179e26363f27 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:46.264472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb124c-a64b-42f3-9f56-179e26363f27 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:46.266263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb124c-a64b-42f3-9f56-179e26363f27 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:46.274612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Mon May 16 18:36:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:36:46.279603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:46.289035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d7c67-da4d-4a42-8513-e4fd07b3fcd7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:46.293399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cdf0a-43cf-485f-812a-80dd1d7c57f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:46.294235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cdf0a-43cf-485f-812a-80dd1d7c57f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:46.294518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cdf0a-43cf-485f-812a-80dd1d7c57f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:46.294796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cdf0a-43cf-485f-812a-80dd1d7c57f2 admin admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:46.296174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cdf0a-43cf-485f-812a-80dd1d7c57f2 admin admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:46.298605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cdf0a-43cf-485f-812a-80dd1d7c57f2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:46.300155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Mon May 16 18:36:46 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:48.550928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32a4a-3939-4f7b-8007-e8f2b1713b39 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:48.551473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32a4a-3939-4f7b-8007-e8f2b1713b39 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:48.551473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32a4a-3939-4f7b-8007-e8f2b1713b39 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:48.552864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:48.558079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:48.558475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:48.558756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:48.600007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d420-74d2-4c25-bb61-4dbe9daf5d38 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:48.600291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d420-74d2-4c25-bb61-4dbe9daf5d38 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:48.600565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1d420-74d2-4c25-bb61-4dbe9daf5d38 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:48.601953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:36:48.607797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:48.608073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:48.608344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:48.652388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG passlib.handlers.bcrypt [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91542) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 16 18:36:48.652826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG passlib.handlers.bcrypt [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91542) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 16 18:36:48.683798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:48.685014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:48.700007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:48.701577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:48.761286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:36:48.784650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:48.797836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:36:48.819594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:48.879274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e579a0e-c0b3-43c1-9467-5d6d516be2d3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:48.911803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:48.912224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126aadf7-6c77-4ae2-b639-0f54f4da89ae None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:48.912600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:48.912918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:48.913195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:48.924449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:36:48.935541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:36:48.936719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:36:48.938168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:36:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:36:48.943153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:48.947109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:48.948009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:48.948293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:48.948570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.011561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.013110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.036248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6295c-c725-48d8-99bb-c2fff4dbe2a6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:49.068738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:49.069402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:49.069728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:49.069957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.070701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:49.071008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.072396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.083726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:49.089272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:49.102709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 18:36:49.108038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:49.192650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2426251-8489-486f-8646-a8f3d17d3d28 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:49.202338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fbcdb-babf-454c-b15a-dd244ae20569 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:49.225185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:49.226044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:49.226325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:49.226600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.227785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.229283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.230893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:36:49 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:49.233007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:49.233790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:49.234600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:49.234870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.236754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:49.336358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecd31a-8c5b-4a7e-aca2-fc898946211b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:49.361998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.363444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.369902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:49.370690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:49.370970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:49.371244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.372422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.373878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.375503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:49.381142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:36:49.383589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:36:49.384542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:36:49.390360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:36:49.390360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:36:49.390360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:36:49.390360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:36:49.390360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:36:49.390691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:49.474729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a566be-285f-4d1b-8e64-0eb3c4d2f6d1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:49.489556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20e0ab-d7e0-41cb-8493-6f54b003edc4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:49.512179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:49.512860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:49.513144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:49.513418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.514254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.domain.id=default, user.id=2ef3d63903f749da901176bdd82045cc) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:49.514568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.516060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e6416-dab7-481d-9b28-24815d073f8a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.521639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:49.522294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:49.522577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:49.522976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:49.523086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 11/20] 10.222.0.36 () {56 vars in 1322 bytes} [Mon May 16 18:36:49 2022] GET /identity/v3/role_assignments?user.id=2ef3d63903f749da901176bdd82045cc&scope.domain.id=default&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:49.523654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:49.524035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:49.525528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16950d-c31c-495c-80c2-3c09bf85c992 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:49.545793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 10/21] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:52.298888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab24ef-e707-4e04-94d5-da8254815ea8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.298888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab24ef-e707-4e04-94d5-da8254815ea8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.298888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab24ef-e707-4e04-94d5-da8254815ea8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.299845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 12/22] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:52.305500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04f333-f4ed-4126-a587-883f65fbaae4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.305918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04f333-f4ed-4126-a587-883f65fbaae4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.306063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04f333-f4ed-4126-a587-883f65fbaae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.329617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6c04f333-f4ed-4126-a587-883f65fbaae4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:52.330812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6c04f333-f4ed-4126-a587-883f65fbaae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:52.408336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 11/23] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:36:52.442945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab24ef-e707-4e04-94d5-da8254815ea8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:52.525338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd5bd-3bd1-4ac2-9cb0-e309f70bfacc None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.526034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd5bd-3bd1-4ac2-9cb0-e309f70bfacc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.526526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd5bd-3bd1-4ac2-9cb0-e309f70bfacc None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.529633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 12/24] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:52.534020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.534514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.534996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.571417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:52.572885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab24ef-e707-4e04-94d5-da8254815ea8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:52.575803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:52.630420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:52.631240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.631526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.631804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.657806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:36:52.665572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:36:52.666386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:36:52.667149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:36:52.673088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 13/25] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:36:52.681987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:52.743862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 13/26] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:36:52.790082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:52.857141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce86f7-12a1-446b-a8b2-7d4533346f50 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:52.928273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:52.928972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.929415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.929849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.930748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:52.931202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:52.932754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eee75a-f41f-4c13-8b23-4a73a34fc2c2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:52.942188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 14/27] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:52.943752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bb9f-fca5-4a97-8c7a-1f85fe90fda9 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.944205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bb9f-fca5-4a97-8c7a-1f85fe90fda9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.944621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6bb9f-fca5-4a97-8c7a-1f85fe90fda9 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.946510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 15/28] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:52.951621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760892f2-50a2-4f44-88dc-494a08cb602f None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:52.952101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760892f2-50a2-4f44-88dc-494a08cb602f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:52.952565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760892f2-50a2-4f44-88dc-494a08cb602f None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:52.954295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 16/29] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:52.960790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760892f2-50a2-4f44-88dc-494a08cb602f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:52.976358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e77b3-9c5b-442e-a79a-41d035e71ec4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.012501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d340-1cc5-427c-a068-b8c384b5d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.013280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d340-1cc5-427c-a068-b8c384b5d1e4 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.013559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d340-1cc5-427c-a068-b8c384b5d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.013832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d340-1cc5-427c-a068-b8c384b5d1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.014638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94d340-1cc5-427c-a068-b8c384b5d1e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.016092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94d340-1cc5-427c-a068-b8c384b5d1e4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.069243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760892f2-50a2-4f44-88dc-494a08cb602f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.103091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c7bec-36b5-4968-abfd-6aec4b41caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.103963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c7bec-36b5-4968-abfd-6aec4b41caf7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.104245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c7bec-36b5-4968-abfd-6aec4b41caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.104522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c7bec-36b5-4968-abfd-6aec4b41caf7 None admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.105778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c7bec-36b5-4968-abfd-6aec4b41caf7 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.107266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c7bec-36b5-4968-abfd-6aec4b41caf7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.109757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 17/30] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:36:52 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:53.110984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.111273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.111549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.127877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:53.129086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:53.151946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 14/31] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 16 18:36:52 2022] POST /identity/v3/roles => generated 219 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:36:53.153850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.154387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.154858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.175429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:53.177636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:53.199192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 18/32] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:36:53.200094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:53.252747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 15/33] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:36:53.264975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:53.335258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67862375-d18a-4c2d-8668-406c0e2c0cd7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.370703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.371423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.371703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.371975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.373453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.375213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.377554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 19/34] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:53.377954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:53.390966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0c089-95f9-4021-be15-9f57061752aa None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.427436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.428623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.429154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.429617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.441620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.443659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:36:53.462933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:36:53.463813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:36:53.464590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:36:53.464590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:36:53.464590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:36:53.464590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:36:53.466778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 16/35] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:36:53.467799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:53.494500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1248e-7157-4254-9b3c-d585c70e8865 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.534818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b66800-bdac-4600-96f3-4cb664ba960b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.535870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b66800-bdac-4600-96f3-4cb664ba960b None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.536347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b66800-bdac-4600-96f3-4cb664ba960b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.536996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b66800-bdac-4600-96f3-4cb664ba960b None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.538484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b66800-bdac-4600-96f3-4cb664ba960b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.540575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b66800-bdac-4600-96f3-4cb664ba960b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.579785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24a828-3f95-4202-85d4-7cf64cb6a752 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.612932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.613799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.614083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.614367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] PATH_INFO: `/v3/domains/default/users/2ef3d63903f749da901176bdd82045cc/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.616809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2ef3d63903f749da901176bdd82045cc, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.618249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.634974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 17/36] 10.222.0.36 () {58 vars in 1361 bytes} [Mon May 16 18:36:53 2022] PUT /identity/v3/domains/default/users/2ef3d63903f749da901176bdd82045cc/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:36:53.636054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:53.677686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 20/37] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 16 18:36:53 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:36:53.679486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489c3f5-75db-45ee-96b0-a8b64c09401f None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.679775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489c3f5-75db-45ee-96b0-a8b64c09401f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.680051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489c3f5-75db-45ee-96b0-a8b64c09401f None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.685599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 21/38] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:53.690215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.690498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.692344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.726428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:53.730297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:53.852002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 22/39] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:36:53.859587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b261a-812e-477f-bbc7-8ca3e15df794 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:53.876456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:53.909389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:53.910343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:53.910640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:53.910922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:53.911949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:53.912265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:53.914366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:53.933142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 18/40] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:53.938668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:54.001574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547bdcd-5c24-4f56-bcc5-66e763facf74 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:54.040880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640cb78-997a-4d9d-85de-af19ec439b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:54.041594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640cb78-997a-4d9d-85de-af19ec439b77 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:54.041874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640cb78-997a-4d9d-85de-af19ec439b77 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:54.042151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640cb78-997a-4d9d-85de-af19ec439b77 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:54.042909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640cb78-997a-4d9d-85de-af19ec439b77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:54.044484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640cb78-997a-4d9d-85de-af19ec439b77 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:54.060054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 23/41] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 16 18:36:53 2022] POST /identity/v3/roles => generated 223 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:36:54.073138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e758cd-fa95-463a-9508-5e898b202e56 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:54.076129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f60b7-5fb3-4ead-baca-0da6ee0d5191 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:54.076419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f60b7-5fb3-4ead-baca-0da6ee0d5191 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:54.076780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f60b7-5fb3-4ead-baca-0da6ee0d5191 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:54.078517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 24/42] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:36:54.084391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:54.084709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:54.084984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:54.107071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:54.108367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:54.114548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:54.115257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:54.115540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:54.115816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:54.116542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:54.118049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:54.176413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 19/43] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:36:53 2022] POST /identity/v3/projects => generated 301 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:36:54.211836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 25/44] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:36:54.236104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:54.348252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073df7-c76d-431e-8203-890a9bc39463 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:54.385307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab8bdd-a5d9-45da-a650-c41dfb3a92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:54.386101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab8bdd-a5d9-45da-a650-c41dfb3a92fa None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:54.386382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab8bdd-a5d9-45da-a650-c41dfb3a92fa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:54.386656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab8bdd-a5d9-45da-a650-c41dfb3a92fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:54.387821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab8bdd-a5d9-45da-a650-c41dfb3a92fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:54.389402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab8bdd-a5d9-45da-a650-c41dfb3a92fa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:54.391548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 20/45] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:54.397198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:54.493094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d2af1-f7fc-4a77-b715-b22ce33dc04e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:54.525362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6b780-6e95-4523-b480-d3b3b7f0ef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:54.526021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6b780-6e95-4523-b480-d3b3b7f0ef6b None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:54.526303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6b780-6e95-4523-b480-d3b3b7f0ef6b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:54.526641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6b780-6e95-4523-b480-d3b3b7f0ef6b None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:54.527359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6b780-6e95-4523-b480-d3b3b7f0ef6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:54.528864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6b780-6e95-4523-b480-d3b3b7f0ef6b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:54.554579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 26/46] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:36:54 2022] POST /identity/v3/projects => generated 312 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:36:56.018685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5c41-b2c5-4937-8e55-26a11f9ca62f None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.019191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5c41-b2c5-4937-8e55-26a11f9ca62f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.019276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5c41-b2c5-4937-8e55-26a11f9ca62f None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.020592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 21/47] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:56.026290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.026707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.026973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.043549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:56.044777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:56.107479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 27/48] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:36:56.130317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5c41-b2c5-4937-8e55-26a11f9ca62f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:56.242637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5c41-b2c5-4937-8e55-26a11f9ca62f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:56.274932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:56.275722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.276009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.276289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.287427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:36:56.296162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:36:56.297069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:36:56.297813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 22/49] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:36:56.303106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:56.395440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4255-3285-4429-8f74-8191f99ea313 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:56.427567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:56.428225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.428505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.428810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.429557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:56.429857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:56.431237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:56.439621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 28/50] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:56.444943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:56.560913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd2ae-0794-4d68-9efd-4de28a11e800 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:56.594592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:56.595492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.595778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.596064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.597327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:56.598819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:56.600723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 23/51] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:36:56 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:56.606134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:56.700151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8db8d-2bb2-4379-9066-b2fd6ac5e76f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:56.730617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d2577-773c-43df-a2f6-eed44519aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:56.731401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2577-773c-43df-a2f6-eed44519aca3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.731679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2577-773c-43df-a2f6-eed44519aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.731947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d2577-773c-43df-a2f6-eed44519aca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.733134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d2577-773c-43df-a2f6-eed44519aca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:56.735075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d2577-773c-43df-a2f6-eed44519aca3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:56.736731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 29/52] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:56.742302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:56.764333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e34ad5-97ea-47e6-9d50-4f3bec662473 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.764647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e34ad5-97ea-47e6-9d50-4f3bec662473 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.764946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e34ad5-97ea-47e6-9d50-4f3bec662473 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.766379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 30/53] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:36:56.771513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.771806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.772090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.788780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:56.789966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:56.839310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65d42c-5160-4ff4-a12a-66bf1beb9a5a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:56.858535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 31/54] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:36:56.873184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:56.873835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:56.874119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:56.874454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:56.875287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.domain.id=default, user.id=2ef3d63903f749da901176bdd82045cc) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:36:56.875601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:56.877062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:56.881700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:56.884823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 24/55] 10.222.0.36 () {56 vars in 1322 bytes} [Mon May 16 18:36:56 2022] GET /identity/v3/role_assignments?user.id=2ef3d63903f749da901176bdd82045cc&scope.domain.id=default&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:56.982068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef6a66-d6b8-4d83-a2fd-57a0779254b1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:57.016546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a34b16-be28-43bf-a2ff-b8a5b4b95d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:57.017381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a34b16-be28-43bf-a2ff-b8a5b4b95d33 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:57.017667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a34b16-be28-43bf-a2ff-b8a5b4b95d33 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:57.017946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a34b16-be28-43bf-a2ff-b8a5b4b95d33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:57.019183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a34b16-be28-43bf-a2ff-b8a5b4b95d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:57.020686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a34b16-be28-43bf-a2ff-b8a5b4b95d33 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:57.022354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 32/56] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:36:57.029134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:57.126959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da42c-466b-485e-a069-8e412f925a9d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:57.158883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc39435-5789-405d-921d-cd86c981f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:57.159528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc39435-5789-405d-921d-cd86c981f185 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:57.159810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc39435-5789-405d-921d-cd86c981f185 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:57.160086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc39435-5789-405d-921d-cd86c981f185 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:57.160813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc39435-5789-405d-921d-cd86c981f185 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:57.162256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc39435-5789-405d-921d-cd86c981f185 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:57.185267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 25/57] 10.222.0.36 () {60 vars in 1140 bytes} [Mon May 16 18:36:57 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:36:59.385306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc0e39-29fa-40b3-be2e-b426b2841cf4 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:59.385817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc0e39-29fa-40b3-be2e-b426b2841cf4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:59.385923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc0e39-29fa-40b3-be2e-b426b2841cf4 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:59.387211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 33/58] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:36:59.392954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:59.393318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:59.393520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:59.409709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:36:59.410886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:36:59.472345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 26/59] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:36:59.496511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc0e39-29fa-40b3-be2e-b426b2841cf4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:59.590065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc0e39-29fa-40b3-be2e-b426b2841cf4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:59.621120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cc411-aa10-4383-959c-693778e3c115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:59.622497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cc411-aa10-4383-959c-693778e3c115 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:59.622497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cc411-aa10-4383-959c-693778e3c115 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:59.622497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cc411-aa10-4383-959c-693778e3c115 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:59.623566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cc411-aa10-4383-959c-693778e3c115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:59.625637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cc411-aa10-4383-959c-693778e3c115 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:59.627209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 34/60] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:36:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:36:59.632620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:36:59.722521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4a0b6-e6d3-4f83-b1fa-c928c608c13f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:36:59.753817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874890a-14a1-4d07-abed-76fd25e77ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:36:59.754459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874890a-14a1-4d07-abed-76fd25e77ed6 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:36:59.754739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874890a-14a1-4d07-abed-76fd25e77ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:36:59.755014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874890a-14a1-4d07-abed-76fd25e77ed6 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:36:59.755704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874890a-14a1-4d07-abed-76fd25e77ed6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:36:59.757175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874890a-14a1-4d07-abed-76fd25e77ed6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:36:59.781710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 27/61] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:36:59 2022] POST /identity/v3/users => generated 275 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:03.063722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b40d04-801b-4a13-8e60-7d1c5688166b None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.064268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b40d04-801b-4a13-8e60-7d1c5688166b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.064268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b40d04-801b-4a13-8e60-7d1c5688166b None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.065740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 35/62] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:03.071801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ece5-411e-4a70-945e-be13e47cef9f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.072278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ece5-411e-4a70-945e-be13e47cef9f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.072348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ece5-411e-4a70-945e-be13e47cef9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.095247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-32d8ece5-411e-4a70-945e-be13e47cef9f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:03.096523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-32d8ece5-411e-4a70-945e-be13e47cef9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:03.184994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 28/63] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:03.222667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b40d04-801b-4a13-8e60-7d1c5688166b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.292934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755f28f-73e6-4d06-8161-b63824cbe29b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.293578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755f28f-73e6-4d06-8161-b63824cbe29b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.294042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755f28f-73e6-4d06-8161-b63824cbe29b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.295857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 29/64] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:03.301537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133956-e99a-45aa-83bf-35ec46ecce73 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.302034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133956-e99a-45aa-83bf-35ec46ecce73 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.302530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133956-e99a-45aa-83bf-35ec46ecce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.321962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3e133956-e99a-45aa-83bf-35ec46ecce73 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:03.323499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3e133956-e99a-45aa-83bf-35ec46ecce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:03.356398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b40d04-801b-4a13-8e60-7d1c5688166b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:03.390604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 30/65] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:03.396734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b604753-0576-41da-b340-48407c5f5571 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.397610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b604753-0576-41da-b340-48407c5f5571 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.398085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b604753-0576-41da-b340-48407c5f5571 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.399440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:03.400082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 31/66] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:03.404341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.404772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.405068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.413943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.414553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.415047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.418009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:03.426637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:03.427631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:03.428377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 36/67] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:03.433218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.447014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:03.448569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:03.517984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 32/68] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:03.518779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.543610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ecaf3-282a-4c1b-8833-6824d23f8ff5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:03.577939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:03.578994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.579418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.579869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.590232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:03.598523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:03.599365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:03.600119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:03.602521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 37/69] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:03.607263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.620807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d87d23-7c0a-4fb1-b42c-5da8cb1d6c97 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:03.655119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:03.656298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.656587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.656883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.657899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:03.658203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:03.659609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50840c-bfb0-4c47-a343-3ccb90371412 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:03.671034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 33/70] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:03.672834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69baeb-7f2b-47d8-a170-d9924165f1e4 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.673124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69baeb-7f2b-47d8-a170-d9924165f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.673396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69baeb-7f2b-47d8-a170-d9924165f1e4 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.674798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 34/71] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:03.675320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69baeb-7f2b-47d8-a170-d9924165f1e4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.716776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649550d3-f117-4f0e-b8ee-ebaffc40299e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:03.752011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:03.752877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.753167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.753447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.764842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:03.773031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:03.773868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:03.774604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 38/72] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:03.775686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.786888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69baeb-7f2b-47d8-a170-d9924165f1e4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:03.821867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:03.822537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.822825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.823106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.823865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:03.824175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:03.825617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec873-22dc-49be-a41e-b00b1bd4488e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:03.833909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 35/73] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:03.835097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.835388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.835669 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.851649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:03.852866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:03.884592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54760145-268b-42ab-ab66-2217ee036b73 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:03.932660 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 36/74] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:03.932972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:03.933087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:03.933170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:03.933170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:03.933170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:03.944503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:03.945998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:03.947674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 39/75] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:03.948429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.016376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7fdcb-d82d-4b4d-a718-dbe3b2c03df4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.041610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a19d8-33c5-4ca6-be68-591f313f3d5f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.048094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.048785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.049072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.049348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.050082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:04.050387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.051753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25f22f-170e-4ee8-9f5d-9742ade2cafc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.059220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 37/76] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.060471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e4be2-3756-4f29-ab97-bc62f2912d67 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.060794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e4be2-3756-4f29-ab97-bc62f2912d67 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.061071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e4be2-3756-4f29-ab97-bc62f2912d67 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.062469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 38/77] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:04.063061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e4be2-3756-4f29-ab97-bc62f2912d67 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.073711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.074575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.074858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.075133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.076322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.078321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.079923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 40/78] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:03 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.080692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.162560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e4be2-3756-4f29-ab97-bc62f2912d67 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.172642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a520685-290c-42e8-a031-f3c3a80761bd None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.194902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029735b-079a-4c5b-a2b5-13f38f6aa160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.195678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029735b-079a-4c5b-a2b5-13f38f6aa160 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.195959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029735b-079a-4c5b-a2b5-13f38f6aa160 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.196230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029735b-079a-4c5b-a2b5-13f38f6aa160 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.204673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.205311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.205594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.205868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.205954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029735b-079a-4c5b-a2b5-13f38f6aa160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a029735b-079a-4c5b-a2b5-13f38f6aa160 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:04.213788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:04.214713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:04.215465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 39/79] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:04.216072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.216580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.216696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.231894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:04.233092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:04.269602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.271066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.295237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 40/80] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:04.295786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.364872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 41/81] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:04.365700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.387511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72d358-85ab-4fbf-9d6b-fa9aab4b2f2d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.419327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.420191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.420474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.420776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.422602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.424034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.425679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 41/82] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.426426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.456989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0e9e-f063-47ec-8652-e8eb8edd1b6e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.489425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.490062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.490347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.490625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.520111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50901fbf-5089-4cb3-9f15-5782a4f5b20d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.554940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.555525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.555807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.556079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.556841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:04.557146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.558521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.562073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.563536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.566414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 42/83] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.567328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.657741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 42/84] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:04.658586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.660094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8eb852-f73d-4340-bfca-df85d56bc375 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.692615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.693417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.693704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.693984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.695156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.696646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.698259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 43/85] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.698259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.750206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6524f-749b-475f-a6ca-f1bcc594ff4d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.783733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.784591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.784908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.785186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.786373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.787814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.789545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 43/86] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.790300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.791972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba20316-caa6-41c1-b4aa-a7c44d69967f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.823369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.823941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.824220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.824494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.881978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8647-ba68-4ff0-95e9-5b375b6ff136 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:04.893635 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.895114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.937313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:04.938191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:04.938548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:04.938841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:04.940087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:04.941584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:04.943221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 44/87] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:04.943977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:04.995328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 44/88] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:04.996155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.038344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0398da-74a5-43b2-9b1a-6f65d026769b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.070137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.070983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.071271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.071555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.072762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.074267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.075915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 45/89] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:04 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:05.076685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.088156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba5c5e-3cf0-4a3d-8d9a-eedd2932a054 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.119400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.120055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.120337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.120614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.121344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.122778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.143219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 45/90] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:04 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:05.144109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.174082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398dd74-ef25-4d04-901a-b8f8802963de None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.210057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.210725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.211005 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.211278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.212111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=a908d4f350ba4702b4193b2939c5a7be) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:05.212419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.213948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.219957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 46/91] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/role_assignments?user.id=a908d4f350ba4702b4193b2939c5a7be&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:05.221292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.259098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b27c1-4a2d-4e02-ad26-fb63baff24fa None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.298703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.299211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.299564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.299934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.348680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398a82e-b699-40dd-b761-5b35766db51d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.375851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.377500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.395245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.396196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.396511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.396934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.398936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.400430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.403592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 47/92] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:05.403592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.509771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 46/93] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:05.510640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.532670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e6179-8d7c-46e0-8980-6e931607fdf2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.573580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.574237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.574523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.574801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.575722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=2ef3d63903f749da901176bdd82045cc) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:05.575993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.577493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.584223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 48/94] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/role_assignments?user.id=2ef3d63903f749da901176bdd82045cc&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:05.584442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.641841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b7e1d-126a-4aab-ad16-28c34316d1a0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.696452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb88ee-c336-4e76-9fbd-01af9afded86 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.705481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.706161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.706449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.706731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.707577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] RBAC: Adding query filter params (role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=a908d4f350ba4702b4193b2939c5a7be) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:05.707892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.709389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.715611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 47/95] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/role_assignments?user.id=a908d4f350ba4702b4193b2939c5a7be&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=d584b6d85205413eb17765aa9c683df4 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:05.734610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ec042-3cee-471a-b5f0-385675d5932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.735558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec042-3cee-471a-b5f0-385675d5932c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.735842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec042-3cee-471a-b5f0-385675d5932c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.736121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec042-3cee-471a-b5f0-385675d5932c None admin] PATH_INFO: `/v3/projects/abae7ed214894390a568ba13aa846f4c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.737486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ec042-3cee-471a-b5f0-385675d5932c None admin] RBAC: Authorizing `identity:get_project(project_id=abae7ed214894390a568ba13aa846f4c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.738980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ec042-3cee-471a-b5f0-385675d5932c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.740830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 49/96] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/projects/abae7ed214894390a568ba13aa846f4c => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:05.747071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:05.863190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4c7f2-a5b7-468e-9dba-19faabd4a702 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:05.911813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:05.912476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:05.912788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:05.913116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:05.913984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=abae7ed214894390a568ba13aa846f4c, user.id=a908d4f350ba4702b4193b2939c5a7be) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:05.914295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:05.915868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7f87e-e237-4696-b34f-81050fef5aa7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:05.922937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 48/97] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:05 2022] GET /identity/v3/role_assignments?user.id=a908d4f350ba4702b4193b2939c5a7be&scope.project.id=abae7ed214894390a568ba13aa846f4c&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:08.494685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec6247-0599-4a87-9e8d-f4b2aa4f15b8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.495292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec6247-0599-4a87-9e8d-f4b2aa4f15b8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.495292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec6247-0599-4a87-9e8d-f4b2aa4f15b8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.496746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 50/98] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:08.502746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb877-00a4-4935-8322-649203edccec None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.503208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb877-00a4-4935-8322-649203edccec None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.503284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fb877-00a4-4935-8322-649203edccec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.527579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8d454-f3a5-4321-a8c6-f43b4eb72911 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.527896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8d454-f3a5-4321-a8c6-f43b4eb72911 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.528142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8d454-f3a5-4321-a8c6-f43b4eb72911 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.529835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 51/99] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:08.530932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a99fb877-00a4-4935-8322-649203edccec None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:08.532382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a99fb877-00a4-4935-8322-649203edccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:08.534745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f929d1a-98de-47d8-86a5-486e23b0c728 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.535026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f929d1a-98de-47d8-86a5-486e23b0c728 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.535300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f929d1a-98de-47d8-86a5-486e23b0c728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.566491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-2f929d1a-98de-47d8-86a5-486e23b0c728 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:08.567686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2f929d1a-98de-47d8-86a5-486e23b0c728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:08.665927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 52/100] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:08.667283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b7f04-7a17-48d2-87c5-e931303bec5d None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.667573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b7f04-7a17-48d2-87c5-e931303bec5d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.667849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b7f04-7a17-48d2-87c5-e931303bec5d None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.672430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 53/101] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:08.688251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.688552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.688846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.701704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 49/102] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:08.702750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb877-00a4-4935-8322-649203edccec None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:08.717883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:08.719248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:08.839277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 54/103] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:08.840540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:08.878633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fb877-00a4-4935-8322-649203edccec None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:08.930770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:08.931731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:08.932208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:08.932549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:08.945688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:08.957807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:08.958737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:08.959575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:08.960698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 50/104] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:08.960698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:08.972935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d47fe5-0fe7-4eea-99cb-cf4741add7ef None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.007527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c6a24-c071-4c49-b03c-4680c4c7aebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.008558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c6a24-c071-4c49-b03c-4680c4c7aebb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.008701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c6a24-c071-4c49-b03c-4680c4c7aebb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.008898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c6a24-c071-4c49-b03c-4680c4c7aebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.010072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c6a24-c071-4c49-b03c-4680c4c7aebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.011543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c6a24-c071-4c49-b03c-4680c4c7aebb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.013211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 55/105] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:09.014385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62a649-eee2-4931-824b-2e43e81fe82a None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.014670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62a649-eee2-4931-824b-2e43e81fe82a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.015256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62a649-eee2-4931-824b-2e43e81fe82a None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.016221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 56/106] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:09.018025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62a649-eee2-4931-824b-2e43e81fe82a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.070233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd56f8e-c16c-47b0-83c5-09c8afcfd172 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.103056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc12434-cc81-45f9-8322-e8ac649161e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.103770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc12434-cc81-45f9-8322-e8ac649161e1 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.104055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc12434-cc81-45f9-8322-e8ac649161e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.104330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc12434-cc81-45f9-8322-e8ac649161e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.114742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc12434-cc81-45f9-8322-e8ac649161e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:09.118769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62a649-eee2-4931-824b-2e43e81fe82a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-4bc12434-cc81-45f9-8322-e8ac649161e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:09.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:09.123133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:09.123889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:09.128436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 51/107] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:09.129632 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020a67-bfef-4af4-b1f9-55b15ee001e0 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.129919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020a67-bfef-4af4-b1f9-55b15ee001e0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.130200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020a67-bfef-4af4-b1f9-55b15ee001e0 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.132674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 52/108] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:09.132674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020a67-bfef-4af4-b1f9-55b15ee001e0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.156878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.158038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.158481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.158754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.159516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:09.159798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.161194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffde685-4f6b-4cc4-834e-3e2c82a84545 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.169497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 57/109] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:09.170648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.170963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.171244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.186791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:09.187959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:09.237611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020a67-bfef-4af4-b1f9-55b15ee001e0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.257059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 58/110] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:09.258475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.272316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be703c93-fca1-4bbc-b6b9-a9a1d18330f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.273234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703c93-fca1-4bbc-b6b9-a9a1d18330f6 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.273676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703c93-fca1-4bbc-b6b9-a9a1d18330f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.274103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703c93-fca1-4bbc-b6b9-a9a1d18330f6 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.274971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be703c93-fca1-4bbc-b6b9-a9a1d18330f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.276569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be703c93-fca1-4bbc-b6b9-a9a1d18330f6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.303727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 53/111] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:37:09 2022] POST /identity/v3/projects => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:09.304857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.305145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.305475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.322593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:09.323808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:09.412844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b892968-4e42-4f6f-b223-3428cc3810f2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.428664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 54/112] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:09.428664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.475452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.476604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.476977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.477257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.478047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:09.478666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.480369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.492521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 59/113] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:09.493290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.556124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03960153-48f1-4c04-b72e-816bfacae940 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.592999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.594119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.594456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.594893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] PATH_INFO: `/v3/users/dc5e354665c940eaa95e44cf8c10a01b` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.600829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b6735-c03b-4d8f-bd5c-b424c41b41f6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.612740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] RBAC: Authorizing `identity:get_user(user_id=dc5e354665c940eaa95e44cf8c10a01b)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.614955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.617288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 55/114] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/users/dc5e354665c940eaa95e44cf8c10a01b => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:09.617886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.633053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.633848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.634131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.634409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.644673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:09.652225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:09.653270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:09.654003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 60/115] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:09.654612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.726710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd6b68-046b-4cc8-9f1f-23cd546e77cc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.757479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bd366-9eb5-41a1-b1f1-b8239084ed24 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.759775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eafe54-ecd8-48bd-a999-569ac48c204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.760567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eafe54-ecd8-48bd-a999-569ac48c204e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.760875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eafe54-ecd8-48bd-a999-569ac48c204e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.761145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eafe54-ecd8-48bd-a999-569ac48c204e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.771720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eafe54-ecd8-48bd-a999-569ac48c204e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-33eafe54-ecd8-48bd-a999-569ac48c204e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:09.779285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:09.780167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:09.781384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 56/116] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:09.782511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdd772-2eda-4506-9ae7-3973c80f6e1c None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.782511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdd772-2eda-4506-9ae7-3973c80f6e1c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.782511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdd772-2eda-4506-9ae7-3973c80f6e1c None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.783958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 57/117] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:09.788320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdd772-2eda-4506-9ae7-3973c80f6e1c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.796899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.797766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.798049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.798325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.799504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.800984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.802593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 61/118] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:09.807254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:09.896219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdd772-2eda-4506-9ae7-3973c80f6e1c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.905009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8280e-1029-4b23-827a-ce6292fc5ea0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:09.934499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.935141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.935420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.935869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.940921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:09.941578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:09.941861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:09.942133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:09.942875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:09.943179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:09.944621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:09.957350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 62/119] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:09.957350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.010132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.011580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.051786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a8d9d-c2b0-461a-8575-5d3094a85158 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.084082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.084768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.085053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.085329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.086069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:10.086371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.087738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4d415-6621-4e4e-9ada-c74fa8db5103 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.095533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 63/120] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.096765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.097056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.097330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.110774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 58/121] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:10.111845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.113635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:10.114803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:10.177391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 64/122] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:10.187760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.212090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7fe02-5759-4348-af46-680953d01f3f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.244382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.245261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.245541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.245811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.246982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.248415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.250829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 59/123] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.251418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.285149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6237cb-f20e-4c2c-9f9d-256fcd4fce78 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.318449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.319322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.319605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.319882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.321113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.322559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.329927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 65/124] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.329927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.352056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7f939-78b9-40a3-9edf-7daf5865b1af None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.384189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.385075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.385356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.385628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.386803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.388242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.389963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 60/125] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.390708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.425467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9899c1a-5120-4377-ac19-ead8c572dcec None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.458372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.459574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.460081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.460473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.461743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.463229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.465115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 66/126] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.469439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.485327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0fa15-42c3-4850-a348-8f47f2b4719d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.518016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.519090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.519368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.519634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.529444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:10.538326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:10.539166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:10.539927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 61/127] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:10.541271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.570015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589557-f31c-4be2-973d-f4960fca1d14 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.603849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.604883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.605239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.605409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888/users/2ef3d63903f749da901176bdd82045cc/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.607401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] RBAC: Authorizing `identity:create_grant(project_id=d32e19faf0104e77a963e4c18b759888, user_id=2ef3d63903f749da901176bdd82045cc, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.609458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.624883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 67/128] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:10 2022] PUT /identity/v3/projects/d32e19faf0104e77a963e4c18b759888/users/2ef3d63903f749da901176bdd82045cc/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:10.626091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.656363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14621b4c-8ed7-45f8-b15c-e4a5d3170ab2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.692134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.693054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.693340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.693620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] PATH_INFO: `/v3/projects/abae7ed214894390a568ba13aa846f4c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.694937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] RBAC: Authorizing `identity:get_project(project_id=abae7ed214894390a568ba13aa846f4c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.696559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.698602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 62/129] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/projects/abae7ed214894390a568ba13aa846f4c => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.705074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.748807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b1d3d-032a-4c7c-b1d4-07d0a89c0c2a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.785068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.786307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.786780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.787242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.788826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.790570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.792880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 68/130] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.793925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.814218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefae9e-1ce3-458b-aa71-94ffca805690 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.852223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.853759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.854315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.854761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.855916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] RBAC: Adding query filter params (role.id=3b8b637b3f194c399eaf71b58f8f9ae6, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=dc5e354665c940eaa95e44cf8c10a01b) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:10.856803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.858717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.865986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 63/131] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/role_assignments?user.id=dc5e354665c940eaa95e44cf8c10a01b&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=3b8b637b3f194c399eaf71b58f8f9ae6 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:10.867307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:10.930835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92120d0-80b9-4fb1-914b-558d17c802e6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:10.982576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:10.984025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:10.984433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:10.984817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:10.985728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:10.986504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:10.987603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:10.996103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e924275-a239-42d8-9d45-8dbce5b6158d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:11.006715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 69/132] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:11.007400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:11.036548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:11.037879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.038370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.038904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] PATH_INFO: `/v3/projects/abae7ed214894390a568ba13aa846f4c/users/a908d4f350ba4702b4193b2939c5a7be/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.041247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] RBAC: Authorizing `identity:create_grant(project_id=abae7ed214894390a568ba13aa846f4c, user_id=a908d4f350ba4702b4193b2939c5a7be, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:11.042967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:11.069699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 64/133] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:10 2022] PUT /identity/v3/projects/abae7ed214894390a568ba13aa846f4c/users/a908d4f350ba4702b4193b2939c5a7be/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:11.071820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:11.163408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8e3c-2714-4b7d-9735-af905e4d747a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:11.199134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:11.200393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.200918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.201383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888/users/a908d4f350ba4702b4193b2939c5a7be/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.204183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] RBAC: Authorizing `identity:create_grant(project_id=d32e19faf0104e77a963e4c18b759888, user_id=a908d4f350ba4702b4193b2939c5a7be, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:11.206010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:11.209988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef813e30-fe76-49ed-8c70-c64b07962a73 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:11.237158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 70/134] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:11 2022] PUT /identity/v3/projects/d32e19faf0104e77a963e4c18b759888/users/a908d4f350ba4702b4193b2939c5a7be/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:11.258572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:11.259861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.260357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.260858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.262342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:11.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:11.266758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 65/135] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:11 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:11.273798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:11.406550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20917d2c-e911-4de7-b3ff-6628613ed74b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:11.444287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca74dad-4949-4ae2-a8be-1c8651c0c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:11.445584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca74dad-4949-4ae2-a8be-1c8651c0c963 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.446060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca74dad-4949-4ae2-a8be-1c8651c0c963 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.446523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca74dad-4949-4ae2-a8be-1c8651c0c963 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.447944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca74dad-4949-4ae2-a8be-1c8651c0c963 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:11.449638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca74dad-4949-4ae2-a8be-1c8651c0c963 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:11.451613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 71/136] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:11 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:11.457874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:11.573850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbb794-79e0-4692-9f62-b925330d9858 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:11.610341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12040fc3-3f8d-4921-a0af-faee0de20b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:11.611574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12040fc3-3f8d-4921-a0af-faee0de20b22 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.612069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12040fc3-3f8d-4921-a0af-faee0de20b22 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.612531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12040fc3-3f8d-4921-a0af-faee0de20b22 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888/users/a908d4f350ba4702b4193b2939c5a7be/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.615938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12040fc3-3f8d-4921-a0af-faee0de20b22 None admin] RBAC: Authorizing `identity:create_grant(project_id=d32e19faf0104e77a963e4c18b759888, user_id=a908d4f350ba4702b4193b2939c5a7be, role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:11.617640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12040fc3-3f8d-4921-a0af-faee0de20b22 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:11.638637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 66/137] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:11 2022] PUT /identity/v3/projects/d32e19faf0104e77a963e4c18b759888/users/a908d4f350ba4702b4193b2939c5a7be/roles/d584b6d85205413eb17765aa9c683df4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:11.878165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f8c77-f6d7-495a-9d2d-43cc09d17497 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.878675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f8c77-f6d7-495a-9d2d-43cc09d17497 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.878745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f8c77-f6d7-495a-9d2d-43cc09d17497 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.880103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 72/138] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:11.894598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:11.895593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:11.896063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:11.919021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:11.920523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:12.011024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 67/139] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:12.048164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f8c77-f6d7-495a-9d2d-43cc09d17497 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:12.171436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f8c77-f6d7-495a-9d2d-43cc09d17497 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:12.218518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b151b-5701-41ed-a37c-5be22b6dbc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:12.219686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b151b-5701-41ed-a37c-5be22b6dbc7b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:12.220179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b151b-5701-41ed-a37c-5be22b6dbc7b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:12.220676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b151b-5701-41ed-a37c-5be22b6dbc7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:12.222073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b151b-5701-41ed-a37c-5be22b6dbc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:12.223724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b151b-5701-41ed-a37c-5be22b6dbc7b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:12.225877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 73/140] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:12.231506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:12.370973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a50523-a5ac-447f-a505-b6c7ebfa52e4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:12.418266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52367cc8-c0c0-43b1-922d-9c33009f749a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:12.419296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367cc8-c0c0-43b1-922d-9c33009f749a None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:12.419775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367cc8-c0c0-43b1-922d-9c33009f749a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:12.420268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367cc8-c0c0-43b1-922d-9c33009f749a None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:12.421799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52367cc8-c0c0-43b1-922d-9c33009f749a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:12.423468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52367cc8-c0c0-43b1-922d-9c33009f749a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:12.457314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 68/141] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:12 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:13.724701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4961c1-952c-43e4-b93f-9644b24dbf3a None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:13.727401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4961c1-952c-43e4-b93f-9644b24dbf3a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:13.727401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4961c1-952c-43e4-b93f-9644b24dbf3a None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:13.727401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 74/142] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:13.733353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:13.734223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:13.734717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:13.754389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:13.755902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:13.839245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 69/143] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:13.871654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4961c1-952c-43e4-b93f-9644b24dbf3a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:14.025532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4961c1-952c-43e4-b93f-9644b24dbf3a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:14.068598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee127d-70ba-4190-980e-62f8cbaf4dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:14.069772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee127d-70ba-4190-980e-62f8cbaf4dcc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.070252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee127d-70ba-4190-980e-62f8cbaf4dcc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.070749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee127d-70ba-4190-980e-62f8cbaf4dcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.085722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee127d-70ba-4190-980e-62f8cbaf4dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-81ee127d-70ba-4190-980e-62f8cbaf4dcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:14.093724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:14.094575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:14.095320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:14.098057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 75/144] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:14.103836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:14.234316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9594-6d65-413b-b2b7-04355172dfcc None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.234995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9594-6d65-413b-b2b7-04355172dfcc None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.235475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9594-6d65-413b-b2b7-04355172dfcc None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.242011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 76/145] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:14.249613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.250545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.251192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.275524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204a8d3-dbee-4e02-906a-7ab8c79997a2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:14.286582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:14.288087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:14.351744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:14.352817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.353298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.353766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.354741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:14.355240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:14.356892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:14.379597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 70/146] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:14.382607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:14.412501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 77/147] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:14.480777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:14.539317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92715552-db21-41f6-a4c8-c5bdfc367f74 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:14.615330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:14.616439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.616981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.617395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] PATH_INFO: `/v3/users/2ef3d63903f749da901176bdd82045cc` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.618740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] RBAC: Authorizing `identity:get_user(user_id=2ef3d63903f749da901176bdd82045cc)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:14.620362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:14.622160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 71/148] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:14 2022] GET /identity/v3/users/2ef3d63903f749da901176bdd82045cc => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:14.627751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:14.630128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec532db-589e-47e7-bd44-73e15cbf1060 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:14.678050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8702d-d828-4df6-b93a-ba858cc700b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:14.678859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8702d-d828-4df6-b93a-ba858cc700b1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.679143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8702d-d828-4df6-b93a-ba858cc700b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.679423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8702d-d828-4df6-b93a-ba858cc700b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.705006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8702d-d828-4df6-b93a-ba858cc700b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-2ed8702d-d828-4df6-b93a-ba858cc700b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:14.712733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:14.713573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:14.714743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 78/149] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:14.715355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4eff3-c256-4e98-a058-4f81c8d2c991 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.715424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4eff3-c256-4e98-a058-4f81c8d2c991 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.715715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4eff3-c256-4e98-a058-4f81c8d2c991 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.719590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 79/150] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:14.726073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4eff3-c256-4e98-a058-4f81c8d2c991 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:14.825748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269e821-b51c-4c41-bc3a-08e4940e01ee None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:14.881431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:14.882333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.882758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.883172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.938501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4eff3-c256-4e98-a058-4f81c8d2c991 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:14.979151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:14.984942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:14.996121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:14.997067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:14.997515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:14.997937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:14.998828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:14.999840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:15.001398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bb923-e428-47da-9bf1-007912476b6d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:15.020345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 80/151] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:15.021876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.022304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.022721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.053824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:15.055331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:15.109923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 72/152] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:15.111019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.193041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 81/153] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:15.193936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.209045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af99a-39e1-490a-a29a-c3c240b513e3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:15.253927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:15.255046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.255464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.255873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] PATH_INFO: `/v3/users/dc5e354665c940eaa95e44cf8c10a01b` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.261369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] RBAC: Authorizing `identity:get_user(user_id=dc5e354665c940eaa95e44cf8c10a01b)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:15.262975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:15.264847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 73/154] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:15 2022] GET /identity/v3/users/dc5e354665c940eaa95e44cf8c10a01b => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:15.265741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.363086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b4c38-4917-422b-9c93-6cfc282a400e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:15.393728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fdc0c-6c93-4c02-8cac-8e18de70c7c6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:15.408580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:15.409624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.410041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.410450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.437853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:15.446550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:15.447563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:15.448492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:15.458323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 74/155] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:15.459919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.474873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb3fa2-ee7a-4f28-9b86-2b5ce33370c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:15.475926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb3fa2-ee7a-4f28-9b86-2b5ce33370c2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.476340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb3fa2-ee7a-4f28-9b86-2b5ce33370c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.476765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb3fa2-ee7a-4f28-9b86-2b5ce33370c2 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.479654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb3fa2-ee7a-4f28-9b86-2b5ce33370c2 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:15.481282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb3fa2-ee7a-4f28-9b86-2b5ce33370c2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:15.483103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 82/156] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:15 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:15.486566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f39e2-8c6b-41ef-91e7-ca8d854d1a1c None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.488354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f39e2-8c6b-41ef-91e7-ca8d854d1a1c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.489559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f39e2-8c6b-41ef-91e7-ca8d854d1a1c None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.490997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 83/157] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:15.493945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7179f9-91c3-4a1f-af1b-493824bf5794 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.495793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7179f9-91c3-4a1f-af1b-493824bf5794 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.496985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7179f9-91c3-4a1f-af1b-493824bf5794 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.498378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 84/158] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:15.501217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7179f9-91c3-4a1f-af1b-493824bf5794 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.621330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec0ab6-0108-4709-9c65-2ea9a36c5d19 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:15.660728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:15.662217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.662505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.662785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.665969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:15.667441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:15.669184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 75/159] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:15 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:15.669953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.681584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7179f9-91c3-4a1f-af1b-493824bf5794 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:15.742310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:15.743001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.743284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.743555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.744320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:15.744641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:15.746066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505aa68-e03d-4b53-9815-331966c7dd3c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:15.762131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 85/160] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:15.763389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.763680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.763958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.800209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:15.804598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:15.841826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2055594-5a41-40ef-85d0-93de79615c67 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:15.899941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:15.900526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.900834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.901170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.902006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=2ef3d63903f749da901176bdd82045cc) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:15.902322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:15.903771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f02e6f-a02b-4552-ae95-71aa94db80c9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:15.909979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 76/161] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:15 2022] GET /identity/v3/role_assignments?user.id=2ef3d63903f749da901176bdd82045cc&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:15.911181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:15.911469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:15.911743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:15.923659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 86/162] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:15.924326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:15.946339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:15.947558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:16.109026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2b9f-3846-4bda-8460-8e21656ad538 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.124968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 77/163] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:16.125828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.151789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.152623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.158676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.158962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888/users/dc5e354665c940eaa95e44cf8c10a01b/roles/3b8b637b3f194c399eaf71b58f8f9ae6` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.163324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] RBAC: Authorizing `identity:create_grant(project_id=d32e19faf0104e77a963e4c18b759888, user_id=dc5e354665c940eaa95e44cf8c10a01b, role_id=3b8b637b3f194c399eaf71b58f8f9ae6)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:16.164870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:16.196025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 87/164] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:15 2022] PUT /identity/v3/projects/d32e19faf0104e77a963e4c18b759888/users/dc5e354665c940eaa95e44cf8c10a01b/roles/3b8b637b3f194c399eaf71b58f8f9ae6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:16.196673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.340502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d54f8-920e-4269-a571-dbfe8b93dd15 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.375754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d0139-0ee6-4905-9fd4-082a8d72ab3c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.390069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.391303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.391774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.392286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.393811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:16.395481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:16.397370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 78/165] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:16.398316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.442896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.443758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.444263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.444807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.471620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:16.479927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:16.485197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:16.486008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:16.488329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 88/166] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:16.489427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.504502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e1f5-3988-4a47-80ae-6dc7b68cae46 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.578197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.579016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.579297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.579571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.604483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:16.614183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:16.615109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:16.615898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 79/167] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:16.616518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.677125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5958f4-7f10-4c9b-9c96-e03cff98306e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.744735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.744735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.744735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.744735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.784764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcde04-0408-407d-943b-587b83e891f0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.818718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.819772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.820252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.820755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.821733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:16.822311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:16.824006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:16.834489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 80/168] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:16.835718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.844465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:16.846026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:16.949978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babf81-a1f1-44ea-b448-8a372f3ca176 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:16.990345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:16.991358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:16.991834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:16.992327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:16.992911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 89/169] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:16.993531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:16.993754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:16.994220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:16.995860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.005739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 81/170] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.006948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.098938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0ddc2-119e-48c4-9948-1677c9a9a7dc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.107009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024bcb4-5f14-4b0c-9c16-1f8983787826 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.133034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.133887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.134161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.134429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.135623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.137333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.138723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 90/171] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:16 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.139468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.147552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faac486-49a4-47e4-aec6-3b7844ce6d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.148788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac486-49a4-47e4-aec6-3b7844ce6d11 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.149267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac486-49a4-47e4-aec6-3b7844ce6d11 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faac486-49a4-47e4-aec6-3b7844ce6d11 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.151233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac486-49a4-47e4-aec6-3b7844ce6d11 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.152919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faac486-49a4-47e4-aec6-3b7844ce6d11 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.156154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 82/172] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.158064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9d317-1f66-4c51-a5b3-bd4518849924 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.158547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9d317-1f66-4c51-a5b3-bd4518849924 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.159029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9d317-1f66-4c51-a5b3-bd4518849924 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.160539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 83/173] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:17.161459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9d317-1f66-4c51-a5b3-bd4518849924 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.241873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f84b7-521d-4deb-8358-dc21ed1bb86a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.267681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9d317-1f66-4c51-a5b3-bd4518849924 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.274330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051581fa-679f-4d71-b62d-e9c59edb8fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.275432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051581fa-679f-4d71-b62d-e9c59edb8fd3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.275432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051581fa-679f-4d71-b62d-e9c59edb8fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.275722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051581fa-679f-4d71-b62d-e9c59edb8fd3 None admin] PATH_INFO: `/v3/users/a908d4f350ba4702b4193b2939c5a7be` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.276923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051581fa-679f-4d71-b62d-e9c59edb8fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=a908d4f350ba4702b4193b2939c5a7be)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.278368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051581fa-679f-4d71-b62d-e9c59edb8fd3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.279985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 91/174] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/users/a908d4f350ba4702b4193b2939c5a7be => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.281768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.282052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.282579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.299215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:17.300388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:17.306352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.307369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.307846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.308340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.366514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 92/175] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:17.367335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.378623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.380476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.464039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bb950-6463-4fd5-ba13-d1b5bd020e59 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.481439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 84/176] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:17.487384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.497990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.498644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.498948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.499747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.500720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=a908d4f350ba4702b4193b2939c5a7be) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:17.500897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.502315 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.508137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 93/177] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/role_assignments?user.id=a908d4f350ba4702b4193b2939c5a7be&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.509012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.592474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bd7ca-23f9-4f28-a945-0c4b5faf6597 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.610998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c9a61-da01-42f6-97e4-4e9f8872535b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.628780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657d764-0a19-4912-80ca-315c2e82de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.629879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657d764-0a19-4912-80ca-315c2e82de1d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.630413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657d764-0a19-4912-80ca-315c2e82de1d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.630893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657d764-0a19-4912-80ca-315c2e82de1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.646406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70a940-343c-4fe4-8ebb-ddb9541f45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.647211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70a940-343c-4fe4-8ebb-ddb9541f45f0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.647525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70a940-343c-4fe4-8ebb-ddb9541f45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.647746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70a940-343c-4fe4-8ebb-ddb9541f45f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.658519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70a940-343c-4fe4-8ebb-ddb9541f45f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1a70a940-343c-4fe4-8ebb-ddb9541f45f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:17.666265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:17.667298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:17.668095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 94/178] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:17.669088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc2f0-13d6-43e3-b5ab-29bdf345044e None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.669365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc2f0-13d6-43e3-b5ab-29bdf345044e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.669649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc2f0-13d6-43e3-b5ab-29bdf345044e None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.671425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 95/179] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:17.672273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc2f0-13d6-43e3-b5ab-29bdf345044e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.712593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657d764-0a19-4912-80ca-315c2e82de1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.714308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657d764-0a19-4912-80ca-315c2e82de1d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.778096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc2f0-13d6-43e3-b5ab-29bdf345044e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.815488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.816360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.816666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.816994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] PATH_INFO: `/v3/projects/abae7ed214894390a568ba13aa846f4c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.818158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] RBAC: Authorizing `identity:get_project(project_id=abae7ed214894390a568ba13aa846f4c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.819668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.821943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 96/180] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/projects/abae7ed214894390a568ba13aa846f4c => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.822776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.824049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 85/181] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:17.825621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.826113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.826581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.851906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:17.853410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:17.925578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 86/182] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:17.926737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:17.928122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c287c-7eca-4602-a2b3-32ee68e769ce None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:17.962512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:17.963197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:17.963474 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:17.963744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:17.964571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:17.964822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:17.966193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:17.974858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 97/183] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:17.975497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.034282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf0af3-e655-42f6-ac98-f187156171c6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.074347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.075399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.075823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.076580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.078492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=abae7ed214894390a568ba13aa846f4c, user.id=a908d4f350ba4702b4193b2939c5a7be) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:18.078979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.079766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cc06d-bc20-4fc1-921a-c4f9eb8ff322 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.080453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.086578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 87/184] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/role_assignments?user.id=a908d4f350ba4702b4193b2939c5a7be&scope.project.id=abae7ed214894390a568ba13aa846f4c&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.087158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.123635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.124989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.125272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.125543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.127397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.129994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.133071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 98/185] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:17 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.134251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.215826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54e34f-5d10-4db2-8784-624d17a0d717 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.266583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658a847-8f53-4e81-b198-d11acfd34548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.268321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658a847-8f53-4e81-b198-d11acfd34548 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.269133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658a847-8f53-4e81-b198-d11acfd34548 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.269707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658a847-8f53-4e81-b198-d11acfd34548 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.273904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd449a2-746f-4088-8a48-d0ebaa0c0b2e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.286067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658a847-8f53-4e81-b198-d11acfd34548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-d658a847-8f53-4e81-b198-d11acfd34548 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:18.297317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:18.298478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:18.299367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 88/186] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:18.300402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17c7c4-eb0c-4417-b0de-31c37da70651 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.301294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17c7c4-eb0c-4417-b0de-31c37da70651 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.301294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17c7c4-eb0c-4417-b0de-31c37da70651 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.303680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 89/187] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:18.303905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17c7c4-eb0c-4417-b0de-31c37da70651 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.330463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.331335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.331612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.331885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] PATH_INFO: `/v3/users/5ae929ebb85a49978c8ebec669b612b0` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.352756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] RBAC: Authorizing `identity:get_user(user_id=5ae929ebb85a49978c8ebec669b612b0)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.355067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.356875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 99/188] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/users/5ae929ebb85a49978c8ebec669b612b0 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.357650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.425512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17c7c4-eb0c-4417-b0de-31c37da70651 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.456705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.457346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.457623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.457891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.458710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] RBAC: Adding query filter params (role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=a908d4f350ba4702b4193b2939c5a7be) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:18.459020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.460432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393deec-88dc-446e-a4f7-976915c71e51 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.461806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222c7b7-5658-442c-b6a3-0474a8374d47 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.466697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 90/189] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/role_assignments?user.id=a908d4f350ba4702b4193b2939c5a7be&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=d584b6d85205413eb17765aa9c683df4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.471793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.471793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.471793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.489214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:18.490805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:18.505269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.506007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.506290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.506568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.507401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:18.507711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.509243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.518906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 100/190] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.519828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.573345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 91/191] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:18.574186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.654087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6f28-7bcb-4181-8120-3ba65e0a3813 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.698988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.699568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.699846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.700116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.706150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c09f6-3836-47f7-9da2-884a3bbb8214 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.751684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.752566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.752871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.753139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] PATH_INFO: `/v3/users/5ae929ebb85a49978c8ebec669b612b0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.755935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] RBAC: Authorizing `identity:get_user(user_id=5ae929ebb85a49978c8ebec669b612b0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.757467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.760110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 92/192] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/users/5ae929ebb85a49978c8ebec669b612b0 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.760110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.781381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.783101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.867100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ef4f0-4dca-4584-be58-9140f7c5eda4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:18.884757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 101/193] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:18.886327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.899748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:18.900538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:18.900846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:18.901120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:18.902285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:18.903739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:18.906245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 93/194] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:18.907069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:18.999417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327daa8-4170-4356-852c-3e208d45049d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.010651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87125-02ac-47a5-a9d7-112f6885fc30 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.034725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.035629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.035917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.036196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.047410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.048264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.048541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.048844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.050026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.051476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.053193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 94/195] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:19.053924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:19.114274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.115782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.152394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e33ea-c76f-4038-93cf-0318ed6fa6f6 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.186104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.186941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.187121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.187348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.188050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.189514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.211179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 95/196] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:19 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:19.211914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:19.212945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 102/197] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:19.225961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:19.326376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b702ef-50ce-44d5-b5d4-74e8a3e4c5d0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.346788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915268a-9339-4857-9447-ecd406ca0a01 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.361668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.362335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.362618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.362900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.363825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=5ae929ebb85a49978c8ebec669b612b0) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:19.364147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.365633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de10d3e-1498-473f-a8c7-3f1db630ff49 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.371503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 96/198] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:19 2022] GET /identity/v3/role_assignments?user.id=5ae929ebb85a49978c8ebec669b612b0&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:19.372868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0d0de-389e-447e-8bc0-13b09947afc9 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.373137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0d0de-389e-447e-8bc0-13b09947afc9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.373421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0d0de-389e-447e-8bc0-13b09947afc9 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.375456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 97/199] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:19.381293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f9824-167d-4732-af2b-9588475f417b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.381636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f9824-167d-4732-af2b-9588475f417b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.381850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f9824-167d-4732-af2b-9588475f417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.393610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.395644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.396471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.397043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.399476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.400363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-436f9824-167d-4732-af2b-9588475f417b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:19.401571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.401755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-436f9824-167d-4732-af2b-9588475f417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:19.404133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 103/200] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:19 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:19.410425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:19.484589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 98/201] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:19.511579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f9824-167d-4732-af2b-9588475f417b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:19.538021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c1d0b-f40e-44d3-9760-1f61cf218131 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.580590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.581275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.581555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.581832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.582670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] RBAC: Adding query filter params (role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=5ae929ebb85a49978c8ebec669b612b0) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:19.582980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.584446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.591517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 104/202] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:19 2022] GET /identity/v3/role_assignments?user.id=5ae929ebb85a49978c8ebec669b612b0&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=d584b6d85205413eb17765aa9c683df4 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:19.635565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f9824-167d-4732-af2b-9588475f417b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.674460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.675271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.675552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.675825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.688156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:19.696143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:19.697119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:19.697916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 99/203] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:19.705096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:19.819163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe490f7e-0244-4e7b-aaa5-d2e207dea0a6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:19.863108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:19.863772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:19.864047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:19.864312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:19.865087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:19.865387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:19.866759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:19.880690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 105/204] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:19.886024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:20.000727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed18f5-d951-4bdb-8420-ff8c6e87e05f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:20.040305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:20.041556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:20.042026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:20.042483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] PATH_INFO: `/v3/users/dc5e354665c940eaa95e44cf8c10a01b` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:20.049362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] RBAC: Authorizing `identity:get_user(user_id=dc5e354665c940eaa95e44cf8c10a01b)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:20.051497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:20.053422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 100/205] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:19 2022] GET /identity/v3/users/dc5e354665c940eaa95e44cf8c10a01b => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:20.059302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:20.183058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988e27-7c5a-4f09-b65a-e8d1c3ba63dc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:20.224419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:20.225097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:20.225378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:20.225652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:20.291437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:20.292924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:20.410660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 106/206] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:20.416217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:20.559874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51544c-b13b-4da9-b98a-54f2be4b2acf None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:20.601545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6501f-043e-420d-8983-462dac055920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:20.602401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6501f-043e-420d-8983-462dac055920 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:20.602683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6501f-043e-420d-8983-462dac055920 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:20.602958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6501f-043e-420d-8983-462dac055920 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:20.609328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6501f-043e-420d-8983-462dac055920 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:20.610884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6501f-043e-420d-8983-462dac055920 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:20.612946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 101/207] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:20 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:20.618570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:20.795496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505671e-1efc-444c-a899-3a3a602257dc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:20.845434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:20.846030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:20.846287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:20.846569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:20.847407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] RBAC: Adding query filter params (role.id=3b8b637b3f194c399eaf71b58f8f9ae6, scope.project.id=d32e19faf0104e77a963e4c18b759888, user.id=dc5e354665c940eaa95e44cf8c10a01b) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:20.847718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:20.849190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fcc11-e0ef-47dc-b9d9-2451a9eea177 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:20.855446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 107/208] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:20 2022] GET /identity/v3/role_assignments?user.id=dc5e354665c940eaa95e44cf8c10a01b&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=3b8b637b3f194c399eaf71b58f8f9ae6 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:21.978391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca01b6f-6369-408f-a4f7-bfff26d37115 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:21.978879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca01b6f-6369-408f-a4f7-bfff26d37115 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:21.978982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca01b6f-6369-408f-a4f7-bfff26d37115 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:21.980583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 102/209] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:21.986219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95511b-c854-47f1-9437-84188f7c8e34 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:21.986643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95511b-c854-47f1-9437-84188f7c8e34 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:21.986809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95511b-c854-47f1-9437-84188f7c8e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.003253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-bb95511b-c854-47f1-9437-84188f7c8e34 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:22.004428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bb95511b-c854-47f1-9437-84188f7c8e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:22.080833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 108/210] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:22.086887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c3c21-6248-422f-ba8f-d954fde668f2 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.087496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c3c21-6248-422f-ba8f-d954fde668f2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.087966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c3c21-6248-422f-ba8f-d954fde668f2 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.089962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 103/211] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:22.096448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2633a4-b79d-4010-8e3a-908a2f144226 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.097082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2633a4-b79d-4010-8e3a-908a2f144226 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.097549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2633a4-b79d-4010-8e3a-908a2f144226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.098791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c3c21-6248-422f-ba8f-d954fde668f2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.115921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ee2633a4-b79d-4010-8e3a-908a2f144226 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:22.117117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ee2633a4-b79d-4010-8e3a-908a2f144226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:22.184374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 109/212] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:22.192812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30710b-ec5d-4532-9329-2b98819faa17 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.192812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30710b-ec5d-4532-9329-2b98819faa17 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.192812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30710b-ec5d-4532-9329-2b98819faa17 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.192812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 110/213] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:22.193066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7be51-d4ca-40bc-a4a1-334aa37f8834 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.193355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7be51-d4ca-40bc-a4a1-334aa37f8834 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.193636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7be51-d4ca-40bc-a4a1-334aa37f8834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.208888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c3c21-6248-422f-ba8f-d954fde668f2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.210774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-06c7be51-d4ca-40bc-a4a1-334aa37f8834 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:22.211892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-06c7be51-d4ca-40bc-a4a1-334aa37f8834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:22.242414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.243206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.243488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.243760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.254242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:22.262055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:22.262961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:22.263738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 104/214] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:22.264420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.278419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 111/215] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:22.280851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09d79-b0af-4f36-9719-b79697521ff8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.281139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09d79-b0af-4f36-9719-b79697521ff8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.281417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09d79-b0af-4f36-9719-b79697521ff8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.283024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 112/216] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:22.283622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09d79-b0af-4f36-9719-b79697521ff8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.370873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4fab1-a055-480e-a518-5c93b8be3fa0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.383002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09d79-b0af-4f36-9719-b79697521ff8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.402904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c41dc-6cec-45e7-8be5-e3341e016510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.403699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c41dc-6cec-45e7-8be5-e3341e016510 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.403978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c41dc-6cec-45e7-8be5-e3341e016510 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.404266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c41dc-6cec-45e7-8be5-e3341e016510 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.414288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c41dc-6cec-45e7-8be5-e3341e016510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:22.414823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.415470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.415748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.416018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.416786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:22.417090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:22.418454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-ab2c41dc-6cec-45e7-8be5-e3341e016510 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:22.422288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:22.423319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:22.424125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 105/217] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:22.424825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.424892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.425162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.428948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 113/218] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:22.429741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.443528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:22.444745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:22.505968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 106/219] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:22.506796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.526049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0deb92-219d-43fc-a791-df3dc85c90c6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.558688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.559476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.559760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.560030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.570487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:22.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:22.579557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:22.580325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 114/220] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:22.580995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.601557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f40461-f54e-4d57-832d-8d40bb2c8d15 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.633090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.633819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.634158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.634430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.635161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:22.635459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:22.636838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:22.644873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 107/221] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:22.645624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.673024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca3003-5b28-4cae-8819-f4ef399d9dfe None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.715083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.715948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.716234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.716507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] PATH_INFO: `/v3/users/aba16173a1934233abbdecee6a6031e8` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.729608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] RBAC: Authorizing `identity:get_user(user_id=aba16173a1934233abbdecee6a6031e8)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:22.731045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:22.732700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 115/222] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/users/aba16173a1934233abbdecee6a6031e8 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:22.733970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.740150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7f4a2-0cdd-4d13-8efc-db6df05b5e91 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.773996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.774791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.775069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.775339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.776504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:22.777967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:22.779569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 108/223] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:22.784359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.826855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a7c1-3dae-4167-b4c9-132dcfd09942 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.857357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.858003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.858767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.859035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.859762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:22.860060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:22.861444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:22.869011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 116/224] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:22.869770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.872429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e96820-eb7a-4139-b322-f0c37f10a966 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.903381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.904225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.904499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:22.905226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] PATH_INFO: `/v3/users/5ae929ebb85a49978c8ebec669b612b0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:22.906409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] RBAC: Authorizing `identity:get_user(user_id=5ae929ebb85a49978c8ebec669b612b0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:22.908106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:22.909761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 109/225] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/users/5ae929ebb85a49978c8ebec669b612b0 => generated 283 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:22.909761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:22.967191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168135-d3fc-48ec-8706-0b39d87c949b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:22.998451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:22.999636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:22.999913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.000181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.002510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297e5f9-b69a-434d-85a5-1786f21fdd1b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.034453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.035039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.035316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.035587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.036279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.037733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.056707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 110/226] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:22 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:23.057734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.066380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.067883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.160111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b1301-59a1-4e10-b96a-a7e26aa85fc7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.171630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 117/227] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:23.176609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.199396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.200276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.200554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.200863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] PATH_INFO: `/v3/users/5ae929ebb85a49978c8ebec669b612b0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.202125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] RBAC: Authorizing `identity:get_user(user_id=5ae929ebb85a49978c8ebec669b612b0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.203570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.209737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 111/228] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:23 2022] GET /identity/v3/users/5ae929ebb85a49978c8ebec669b612b0 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:23.209737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.272278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e84e29-2940-4b1f-b723-639e1f8741d6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.306840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.307626 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.307934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.308175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.309556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.311090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.313421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 118/229] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:23 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:23.314181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.325629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833831-d179-4844-990a-8a6ac365ce9e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.363583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.364594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.365005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.365384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.366703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.368273 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.370476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 112/230] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:23 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:23.371314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.416058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcac3e1-2893-44fd-93ae-9f2fb7ce9594 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.450060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.450924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.451197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.451464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.452722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.454167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.456148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 119/231] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:23 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:23.460653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.482110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11567f50-e38f-4fe5-ad50-6a012f834537 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.518624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.519725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.520025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.520285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714/users/5ae929ebb85a49978c8ebec669b612b0/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.522789 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] RBAC: Authorizing `identity:create_grant(project_id=9326b1f4ec284857a398c6df75671714, user_id=5ae929ebb85a49978c8ebec669b612b0, role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.524244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.542624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 113/232] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:23 2022] PUT /identity/v3/projects/9326b1f4ec284857a398c6df75671714/users/5ae929ebb85a49978c8ebec669b612b0/roles/d584b6d85205413eb17765aa9c683df4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:23.543339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:23.563055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad40337-9b1d-46f3-9cf4-8cedadb92098 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.611007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.611728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.612155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.612433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.613609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=aba16173a1934233abbdecee6a6031e8) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:23.613925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.615394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612903e5-362a-47f3-b3d2-018d157aa918 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.621870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 120/233] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:23 2022] GET /identity/v3/role_assignments?user.id=aba16173a1934233abbdecee6a6031e8&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:23.691153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474fbef-9499-4670-8465-b54de2027389 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:23.759133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4455e8-3334-48a0-8e06-6ae3588ee86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:23.760052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4455e8-3334-48a0-8e06-6ae3588ee86a None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:23.760326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4455e8-3334-48a0-8e06-6ae3588ee86a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:23.760599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4455e8-3334-48a0-8e06-6ae3588ee86a None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714/users/5ae929ebb85a49978c8ebec669b612b0/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:23.766116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4455e8-3334-48a0-8e06-6ae3588ee86a None admin] RBAC: Authorizing `identity:create_grant(project_id=9326b1f4ec284857a398c6df75671714, user_id=5ae929ebb85a49978c8ebec669b612b0, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:23.766578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4455e8-3334-48a0-8e06-6ae3588ee86a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:23.799244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 114/234] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:23 2022] PUT /identity/v3/projects/9326b1f4ec284857a398c6df75671714/users/5ae929ebb85a49978c8ebec669b612b0/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:25.918926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274eacc-123d-4096-8f66-7be00da5646f None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:25.919937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274eacc-123d-4096-8f66-7be00da5646f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:25.920424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274eacc-123d-4096-8f66-7be00da5646f None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:25.922391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 121/235] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:25.927551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:25.928001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:25.928073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:25.949451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a25d-3c30-4c07-b30d-4d30bb4472a4 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:25.949770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a25d-3c30-4c07-b30d-4d30bb4472a4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:25.949997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a25d-3c30-4c07-b30d-4d30bb4472a4 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:25.951486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 122/236] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:37:25.952802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:25.953981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:25.966166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:25.966745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:25.967223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.001621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:26.003146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:26.076360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 115/237] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:26.099239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 123/238] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:26.111681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.123239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.298313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede85f6-8c3d-4ce3-8ab1-5b73d923a1b7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:26.313853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d379e-a682-4800-97f8-ad76d78b0b8e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:26.367253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20f4ef-162a-431d-b8e4-8b92534f5b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:26.368061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f4ef-162a-431d-b8e4-8b92534f5b81 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.368339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f4ef-162a-431d-b8e4-8b92534f5b81 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.368612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f4ef-162a-431d-b8e4-8b92534f5b81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.374454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63f4e6-b9b2-4e72-b23d-ff9ca3e673cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:26.375254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63f4e6-b9b2-4e72-b23d-ff9ca3e673cd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.375532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63f4e6-b9b2-4e72-b23d-ff9ca3e673cd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.375803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63f4e6-b9b2-4e72-b23d-ff9ca3e673cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.379638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20f4ef-162a-431d-b8e4-8b92534f5b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:26.381393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63f4e6-b9b2-4e72-b23d-ff9ca3e673cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:26.382855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63f4e6-b9b2-4e72-b23d-ff9ca3e673cd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:26.384691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 116/239] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:26.385794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a59af2-17cb-47a2-8fc5-9c9b122fa088 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.386086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a59af2-17cb-47a2-8fc5-9c9b122fa088 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.386357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a59af2-17cb-47a2-8fc5-9c9b122fa088 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-ee20f4ef-162a-431d-b8e4-8b92534f5b81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:26.387493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:26.388309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:26.390124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:26.390725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 117/240] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:26.390792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 124/241] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:26.393800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a59af2-17cb-47a2-8fc5-9c9b122fa088 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.395310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac5485-c9c1-46ed-ada5-8c16a8e023da None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.395668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac5485-c9c1-46ed-ada5-8c16a8e023da None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.395955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac5485-c9c1-46ed-ada5-8c16a8e023da None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.398035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 125/242] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:26.398947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.399234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.399510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.424155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:26.425391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:26.495574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 126/243] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:26.496921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.512237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a59af2-17cb-47a2-8fc5-9c9b122fa088 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:26.546089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d474bb-a432-4279-abda-d80c5f275b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:26.546747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d474bb-a432-4279-abda-d80c5f275b99 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.547020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d474bb-a432-4279-abda-d80c5f275b99 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.547284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d474bb-a432-4279-abda-d80c5f275b99 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.547977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d474bb-a432-4279-abda-d80c5f275b99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:26.549482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d474bb-a432-4279-abda-d80c5f275b99 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:26.570815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 118/244] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:26 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:26.571869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.572153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.572425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.593543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:26.594829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:26.613950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9febf-3313-4e94-b82e-8ac1664af431 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:26.660925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:26.661991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.662543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.662818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.663752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:26.664361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:26.666470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dab31-4b30-4afc-a350-6e1050a64cd8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:26.674546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 119/245] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:26.674546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.676086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 127/246] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:26.677349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95035-b480-433c-b21f-eea922694f95 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.677632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95035-b480-433c-b21f-eea922694f95 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.677905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95035-b480-433c-b21f-eea922694f95 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.680683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 128/247] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:26.686106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95035-b480-433c-b21f-eea922694f95 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.798444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b52b-2c8d-4847-bfc1-8fdcf68ed385 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:26.811883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95035-b480-433c-b21f-eea922694f95 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:26.837818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b7b9d-f351-4c41-82bd-bf300b53c4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:26.838910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7b9d-f351-4c41-82bd-bf300b53c4a7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.839181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7b9d-f351-4c41-82bd-bf300b53c4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.839444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7b9d-f351-4c41-82bd-bf300b53c4a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.852938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b7b9d-f351-4c41-82bd-bf300b53c4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:26.853782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:26.856894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.856894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.856894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] PATH_INFO: `/v3/users/7e3563976060470089c252079ed8b72e` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-652b7b9d-f351-4c41-82bd-bf300b53c4a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:26.862418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:26.863286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:26.864067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:26.864717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 120/248] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:26.865885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:26.866389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:26.866672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:26.869287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] RBAC: Authorizing `identity:get_user(user_id=7e3563976060470089c252079ed8b72e)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:26.871056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:26.873123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 129/249] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/users/7e3563976060470089c252079ed8b72e => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:26.875199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.888842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:26.890678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:26.971475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 121/250] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:26.972041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:26.989127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60298f24-3300-4f9b-b149-478c6786fb3a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.023853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.024665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.024952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.025221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.036160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:27.043952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:27.044862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:27.045701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 130/251] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:27.046338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.084765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb6091-bf59-465e-a371-f9082b0f1a96 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.122453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.123127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.123408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.124204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.124997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:27.125330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.126673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.135805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 122/252] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.136749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.147943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916cb0a-3723-4786-ae71-6cd1d303401a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.182004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.182659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.182939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.183210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.236776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca92c58-46e0-4d46-9e80-aae03aa5cb15 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.251760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.253269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.270964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.271759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.272039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.272314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.282916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:27.290608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:27.292074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 123/253] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:27.293160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.361389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 131/254] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:27.362384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.409931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b58e-e789-44b8-a30f-5ebbeaf62cfc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.456926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.457594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.457871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.458143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.458890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:27.459194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.460556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.476800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 124/255] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.478755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.492792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7be2df-a4ff-4665-9fda-e7f460fa30c8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.537793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.539210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.539797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.540563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] PATH_INFO: `/v3/users/5ae929ebb85a49978c8ebec669b612b0` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.542296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] RBAC: Authorizing `identity:get_user(user_id=5ae929ebb85a49978c8ebec669b612b0)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.544129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.547293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 132/256] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/users/5ae929ebb85a49978c8ebec669b612b0 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.548236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.588319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acaa9d-3994-409d-911c-1773e14af317 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.627669 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.628210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.628558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.628931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.629869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:27.630050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.631520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.640502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 125/257] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.641568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.652221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab74160-660f-4e92-9178-9bc99927e62f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.686972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.688113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.688593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.689093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.690461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.692124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.698799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 133/258] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.699834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.742937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fed816-d900-4df7-ac19-7f604f00d5dd None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.776813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.777681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.778005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.778279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] PATH_INFO: `/v3/users/aba16173a1934233abbdecee6a6031e8` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.779467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] RBAC: Authorizing `identity:get_user(user_id=aba16173a1934233abbdecee6a6031e8)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.781462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.783407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 126/259] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/users/aba16173a1934233abbdecee6a6031e8 => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.784239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:27.805880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60675022-bda5-4e73-9e43-60a1cc4eef93 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.839193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.840180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.840682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.841159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.885111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d5d1f-c2d3-4006-b945-6bc0157c9c9a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:27.913219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.915121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.922058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:27.922955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:27.923247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:27.923498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] PATH_INFO: `/v3/users/5ae929ebb85a49978c8ebec669b612b0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:27.924858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] RBAC: Authorizing `identity:get_user(user_id=5ae929ebb85a49978c8ebec669b612b0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:27.926332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:27.928416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 127/260] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/users/5ae929ebb85a49978c8ebec669b612b0 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:27.929338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.030857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 134/261] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:28.031845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.044587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2198b03-1057-4183-ba6c-33cc0471f8de None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.086489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.087294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.087889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.088168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.089305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] RBAC: Adding query filter params (role.id=3b8b637b3f194c399eaf71b58f8f9ae6, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=7e3563976060470089c252079ed8b72e) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:28.089624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.091787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.099643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 128/262] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:27 2022] GET /identity/v3/role_assignments?user.id=7e3563976060470089c252079ed8b72e&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=3b8b637b3f194c399eaf71b58f8f9ae6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:28.100509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.132938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e8800-a146-4837-85d1-b3dc4e4fda49 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.169149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.170278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.170684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.171026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.172753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.174606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.177396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 135/263] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:28 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:28.178254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.281022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0316812-cdbb-42de-bc4a-4e8d85e049de None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.351986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.352644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.352926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.353196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.362479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6e8d-2765-45b9-932f-b71c86452d5b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.396784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.397640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.397914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.398182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.399473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.400996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.402694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 136/264] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:28 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:28.409266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.424777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.426547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.524544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 129/265] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:28.525736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.565082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ca055-5a65-45a3-a912-f5d57dae1d52 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.629551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e1c2-2a07-463e-9a5d-545aa281547b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.640293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.641208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.641486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.641754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714/users/aba16173a1934233abbdecee6a6031e8/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.645741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] RBAC: Authorizing `identity:create_grant(project_id=9326b1f4ec284857a398c6df75671714, user_id=aba16173a1934233abbdecee6a6031e8, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.647202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.678011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 137/266] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:28 2022] PUT /identity/v3/projects/9326b1f4ec284857a398c6df75671714/users/aba16173a1934233abbdecee6a6031e8/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:28.678855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.693694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.694346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.694626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.694895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.695738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] RBAC: Adding query filter params (role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=5ae929ebb85a49978c8ebec669b612b0) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:28.696050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.697541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.709320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 130/267] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:28 2022] GET /identity/v3/role_assignments?user.id=5ae929ebb85a49978c8ebec669b612b0&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=d584b6d85205413eb17765aa9c683df4 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:28.813498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec66f9b-a9f5-4b5a-a042-eb57142a9440 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:28.848468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254b739-e461-4e8d-9296-e9512195ce81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:28.849381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254b739-e461-4e8d-9296-e9512195ce81 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:28.849657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254b739-e461-4e8d-9296-e9512195ce81 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:28.849927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254b739-e461-4e8d-9296-e9512195ce81 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:28.851233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254b739-e461-4e8d-9296-e9512195ce81 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:28.852780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254b739-e461-4e8d-9296-e9512195ce81 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:28.854672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 138/268] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:28 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:28.861320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:28.976118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d61b-ea33-4eed-bd9e-f1bc9021f428 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:29.013794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:29.014568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.014853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.015123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.015967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=5ae929ebb85a49978c8ebec669b612b0) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:29.016276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:29.017831 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacef42-37a8-445b-9f1a-84e47e62b309 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:29.024733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 131/269] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:28 2022] GET /identity/v3/role_assignments?user.id=5ae929ebb85a49978c8ebec669b612b0&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:29.205341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a56af-269a-4a4d-9d0c-6161bf9cba06 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.205850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a56af-269a-4a4d-9d0c-6161bf9cba06 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.206015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a56af-269a-4a4d-9d0c-6161bf9cba06 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.207485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 139/270] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:29.213706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.214202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.214202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.235375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:29.236596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:29.314808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 132/271] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:29.335695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d184-64fd-4063-b67b-accf0e32fc62 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.335991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d184-64fd-4063-b67b-accf0e32fc62 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.336307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3d184-64fd-4063-b67b-accf0e32fc62 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.337763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 140/272] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:29.339921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:29.348619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.348619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.348619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.371486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:29.373055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:29.447378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 141/273] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:29.459886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8f4c7-32ae-428f-9fb7-396e5f6342e4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:29.473587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:29.499140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:29.499939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.500206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.500472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.515890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:29.525444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:29.526320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:29.527098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 133/274] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:29.535208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:29.605297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6031-e813-4672-a003-c5ed0f998b38 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:29.642006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:29.645382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.645878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.646344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.647739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:29.649459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:29.651038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fa41f-a82f-4cd3-87fd-283ae997c15c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:29.651510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 142/275] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:29.657774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:29.693603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:29.694264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.694543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.694817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.695584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:29.696013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:29.698192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:29.712665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 134/276] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:29.714764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:29.775452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e43dd0-6e7d-4596-ba1e-4963195c1407 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:29.808893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:29.809608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.809885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.810156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.810864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:29.812888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:29.843306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ef7e2-80c6-4a41-b569-72401fe9c238 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:29.844673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 143/277] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:29 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:37:29.889050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58436c4-a836-4a30-badf-d097da9d33e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:29.890850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58436c4-a836-4a30-badf-d097da9d33e6 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:29.891357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58436c4-a836-4a30-badf-d097da9d33e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:29.891834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58436c4-a836-4a30-badf-d097da9d33e6 None admin] PATH_INFO: `/v3/users/5f96eac557c040c5a47e721aa88cfddc` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:29.908365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58436c4-a836-4a30-badf-d097da9d33e6 None admin] RBAC: Authorizing `identity:get_user(user_id=5f96eac557c040c5a47e721aa88cfddc)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:29.910254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58436c4-a836-4a30-badf-d097da9d33e6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:29.912318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 135/278] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:29 2022] GET /identity/v3/users/5f96eac557c040c5a47e721aa88cfddc => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:29.921943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:30.034552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3995f4-6657-47b3-a8a6-2f823a3bb665 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:30.069863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:30.070518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:30.070797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:30.071125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:30.071894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, user.id=5f96eac557c040c5a47e721aa88cfddc, scope.system=all) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:30.072198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:30.073664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374af1-447d-4989-aabb-76d979d34bba None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:30.079613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 144/279] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 16 18:37:29 2022] GET /identity/v3/role_assignments?user.id=5f96eac557c040c5a47e721aa88cfddc&scope.system=all&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:30.941700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e41c21-0b40-47eb-8a42-713c8e61055e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:30.942218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e41c21-0b40-47eb-8a42-713c8e61055e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:30.942293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e41c21-0b40-47eb-8a42-713c8e61055e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:30.943858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 136/280] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:30.949440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:30.949863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:30.950046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:30.967731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:30.969093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:31.037526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 145/281] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:31.060152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e41c21-0b40-47eb-8a42-713c8e61055e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:31.194024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e41c21-0b40-47eb-8a42-713c8e61055e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:31.237472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:31.238259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:31.238541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:31.238814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:31.253573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:31.261148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:31.262016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:31.263215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 137/282] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:31.268082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:31.378419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a383b4-37d4-46c8-9d86-f8d881b1ca8c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:31.412594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:31.413311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:31.413575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:31.413846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:31.414588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:31.414889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:31.416282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:31.426183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 146/283] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:31.432106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:31.563715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca96f3-c097-4d5b-bb89-575dde9f1700 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:31.606018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbab78-f0a2-4e02-b0f1-6b40eff0a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:31.606894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbab78-f0a2-4e02-b0f1-6b40eff0a77e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:31.607168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbab78-f0a2-4e02-b0f1-6b40eff0a77e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:31.607436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbab78-f0a2-4e02-b0f1-6b40eff0a77e None admin] PATH_INFO: `/v3/users/7e3563976060470089c252079ed8b72e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:31.609195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbab78-f0a2-4e02-b0f1-6b40eff0a77e None admin] RBAC: Authorizing `identity:get_user(user_id=7e3563976060470089c252079ed8b72e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:31.610755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbab78-f0a2-4e02-b0f1-6b40eff0a77e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:31.612434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 138/284] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:31 2022] GET /identity/v3/users/7e3563976060470089c252079ed8b72e => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:31.617825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:31.652941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f3be5-600e-4a14-a702-75b808c9a02f None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:31.654314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f3be5-600e-4a14-a702-75b808c9a02f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:31.654314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f3be5-600e-4a14-a702-75b808c9a02f None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:31.656671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 139/285] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:37:31.677948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:31.678233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:31.678495 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:31.703988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:31.705194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:31.742679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137d41-8507-4e61-b653-b3fc5dfe5a2b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:31.782722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:31.784456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:31.787516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:31.788001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:31.793701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:31.795361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:31.798563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 147/286] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:31 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:31.802634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:31.837661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 140/287] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:31.865604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:31.976473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe26ebe-575b-47e5-8534-f006a78b542d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:32.014876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:32.016039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.016703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.018211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.023265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a679e5-e50b-4e41-90c8-7a4ceb44d6df None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:32.040567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:32.049544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:32.050439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:32.051659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 141/288] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:32.064662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:32.109066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:32.109870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.110147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.110414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714/users/7e3563976060470089c252079ed8b72e/roles/3b8b637b3f194c399eaf71b58f8f9ae6` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.112418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9326b1f4ec284857a398c6df75671714, user_id=7e3563976060470089c252079ed8b72e, role_id=3b8b637b3f194c399eaf71b58f8f9ae6)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:32.122037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:32.157223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 148/289] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:37:31 2022] PUT /identity/v3/projects/9326b1f4ec284857a398c6df75671714/users/7e3563976060470089c252079ed8b72e/roles/3b8b637b3f194c399eaf71b58f8f9ae6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:32.178714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f34b4a-eb57-4fa2-bca1-71199528139e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:32.224224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:32.224929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.225213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.225489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.226242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:32.226555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:32.228051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:32.237320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 142/290] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:32.243413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:32.397165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5acf75-0819-4250-83df-1ec165b9b3c5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:32.434862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8817d-252d-48a7-905e-c283d787141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:32.435775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8817d-252d-48a7-905e-c283d787141e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.436120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8817d-252d-48a7-905e-c283d787141e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.436905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8817d-252d-48a7-905e-c283d787141e None admin] PATH_INFO: `/v3/users/aba16173a1934233abbdecee6a6031e8` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.439423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8817d-252d-48a7-905e-c283d787141e None admin] RBAC: Authorizing `identity:get_user(user_id=aba16173a1934233abbdecee6a6031e8)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:32.440921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8817d-252d-48a7-905e-c283d787141e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:32.442915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 149/291] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:32 2022] GET /identity/v3/users/aba16173a1934233abbdecee6a6031e8 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:32.448347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:32.554441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0665-dc86-472c-8edb-e4ef6b0f14da None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:32.597543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:32.598231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.598505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.598782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.680235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:32.681642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:32.762274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e69ddb-4c71-4d44-a208-f0486513ecf4 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.762597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e69ddb-4c71-4d44-a208-f0486513ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.762832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e69ddb-4c71-4d44-a208-f0486513ecf4 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.764421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 150/292] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:37:32.769599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424d4c8-7ace-4147-b8fb-20169c2365d7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.769881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424d4c8-7ace-4147-b8fb-20169c2365d7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.770153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424d4c8-7ace-4147-b8fb-20169c2365d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.803620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 143/293] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 18:37:32.805660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1424d4c8-7ace-4147-b8fb-20169c2365d7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:32.806857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1424d4c8-7ace-4147-b8fb-20169c2365d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:32.809605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:32.904138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 151/294] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:32.906537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3867b-1d3a-42a9-8124-7c81b04f24e6 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.906826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3867b-1d3a-42a9-8124-7c81b04f24e6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.907101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3867b-1d3a-42a9-8124-7c81b04f24e6 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.908674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 152/295] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:32.913918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.914273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.914622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.938350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fc64-7785-4483-87ec-d2430b5e6a94 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:32.942625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:32.943880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:32.973773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a5f07-53b4-4d7f-83b8-5bd224d01133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:32.974630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a5f07-53b4-4d7f-83b8-5bd224d01133 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.974905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a5f07-53b4-4d7f-83b8-5bd224d01133 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.975172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a5f07-53b4-4d7f-83b8-5bd224d01133 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.976345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a5f07-53b4-4d7f-83b8-5bd224d01133 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:32.977810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a5f07-53b4-4d7f-83b8-5bd224d01133 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:32.980742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 144/296] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:32 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:32.980988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8cfd-ffee-4b17-bdd6-eed2d361fd64 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:32.981269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8cfd-ffee-4b17-bdd6-eed2d361fd64 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:32.981543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8cfd-ffee-4b17-bdd6-eed2d361fd64 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:32.983528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 145/297] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:32.983984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f8cfd-ffee-4b17-bdd6-eed2d361fd64 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.016687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 153/298] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:33.017375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.091955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f8cfd-ffee-4b17-bdd6-eed2d361fd64 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.114006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb24cc-f339-4eef-b764-2d674440c01e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.130002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69279fd0-70db-4e45-a0bd-59f9a0430935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.130788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69279fd0-70db-4e45-a0bd-59f9a0430935 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.131063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69279fd0-70db-4e45-a0bd-59f9a0430935 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.131328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69279fd0-70db-4e45-a0bd-59f9a0430935 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.142078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69279fd0-70db-4e45-a0bd-59f9a0430935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:33.146313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.146961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.147242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.147512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.148336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=aba16173a1934233abbdecee6a6031e8) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:33.148665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-69279fd0-70db-4e45-a0bd-59f9a0430935 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:33.149752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:33.150604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:33.151367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:33.151942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.152007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 146/299] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:33.152076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.152590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.152712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.168908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 154/300] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/role_assignments?user.id=aba16173a1934233abbdecee6a6031e8&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.168908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.183232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:33.184961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:33.273744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9aa14-67dd-4107-9c4e-300218dca8b3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.280676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 147/301] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:33.284654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.334607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.336450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.337013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.337477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.352153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:33.361215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:33.362100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:33.362865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 155/302] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:33.363488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.417099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaace5-2962-49dd-87af-01ba41f01818 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.459071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.459726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.459998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.460266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.462182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] RBAC: Adding query filter params (name=reader) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:33.462486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.466490 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.474620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 148/303] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.474620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.487647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc289d7-c201-4624-b0e0-00aa03836acc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.521451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.522221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.522499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.522768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.523930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.525380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.531422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 156/304] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.531422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.582273 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8f8e3-8b6f-47f9-b051-331d7aeb609a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.614187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.614842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.615118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.615388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.616116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:33.616415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.617797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.626067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 149/305] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.628930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.633799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0eb7-7fe2-4e22-9da4-ec5c9facd770 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.673014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.674479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.675078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.675829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] PATH_INFO: `/v3/users/3ad58b815f664028937d0ea3884ea30f` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.691680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] RBAC: Authorizing `identity:get_user(user_id=3ad58b815f664028937d0ea3884ea30f)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.693042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.694900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 157/306] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/users/3ad58b815f664028937d0ea3884ea30f => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.695656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.728297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d959c0b-bf79-48b3-80a7-b9f179c4a9d3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.760944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.761605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.761887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.762162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] PATH_INFO: `/v3/groups` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.762843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.764988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.779609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 150/307] 10.222.0.36 () {60 vars in 1134 bytes} [Mon May 16 18:37:33 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:33.780366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.799907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c15c2-624d-439c-8295-eed0c0ca43bf None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.833851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.834839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.835118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.835386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] PATH_INFO: `/v3/users/5f96eac557c040c5a47e721aa88cfddc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.836734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] RBAC: Authorizing `identity:get_user(user_id=5f96eac557c040c5a47e721aa88cfddc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.838368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.840554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 158/308] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/users/5f96eac557c040c5a47e721aa88cfddc => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.845723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:33.899665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7f68-9036-4e7f-a000-c599bf6c0eea None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:33.934488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:33.935213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:33.935503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:33.935778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:33.937268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] RBAC: Adding query filter params (role.id=3b8b637b3f194c399eaf71b58f8f9ae6, user.id=3ad58b815f664028937d0ea3884ea30f, scope.system=all) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:33.937586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:33.939060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe758e-cc66-439a-ac10-b4826ccaa8fa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:33.945379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 151/309] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 16 18:37:33 2022] GET /identity/v3/role_assignments?user.id=3ad58b815f664028937d0ea3884ea30f&scope.system=all&role.id=3b8b637b3f194c399eaf71b58f8f9ae6 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:33.975192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3f979-7191-47dd-be48-79b0099c613e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:34.015683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3257-c3b6-4b60-8f92-8c214dc84170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:34.016557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3257-c3b6-4b60-8f92-8c214dc84170 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:34.016867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3257-c3b6-4b60-8f92-8c214dc84170 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:34.017137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3257-c3b6-4b60-8f92-8c214dc84170 None admin] PATH_INFO: `/v3/system/users/5f96eac557c040c5a47e721aa88cfddc/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:34.018669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3257-c3b6-4b60-8f92-8c214dc84170 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5f96eac557c040c5a47e721aa88cfddc, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:34.020069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3257-c3b6-4b60-8f92-8c214dc84170 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:34.031982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 159/310] 10.222.0.36 () {58 vars in 1334 bytes} [Mon May 16 18:37:33 2022] PUT /identity/v3/system/users/5f96eac557c040c5a47e721aa88cfddc/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:37:34.728974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0818bd-460c-4371-ad74-9e4260812dbf None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:34.729516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0818bd-460c-4371-ad74-9e4260812dbf None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:34.729516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0818bd-460c-4371-ad74-9e4260812dbf None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:34.731041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 152/311] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:34.736506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:34.737352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:34.737838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:34.754910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:34.756084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:34.823501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 160/312] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:34.847083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0818bd-460c-4371-ad74-9e4260812dbf None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:34.947559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0818bd-460c-4371-ad74-9e4260812dbf None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:34.982253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:34.983021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:34.983300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:34.983573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:34.995260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:35.003783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:35.004671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:35.005449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:35.009744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 153/313] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:35.013729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:35.118011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3dca-bedd-4c5d-ab01-435fc5536e31 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:35.152693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:35.153668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:35.154146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:35.154620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:35.155612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:35.156091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:35.157932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:35.172046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 161/314] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:35.173858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:35.274790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca5612-a92e-41bb-9f1c-b096700cb5db None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:35.307904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856276e-7173-496a-895c-df189fed816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:35.309099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856276e-7173-496a-895c-df189fed816e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:35.309562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856276e-7173-496a-895c-df189fed816e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:35.310029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856276e-7173-496a-895c-df189fed816e None admin] PATH_INFO: `/v3/users/7e3563976060470089c252079ed8b72e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:35.311906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856276e-7173-496a-895c-df189fed816e None admin] RBAC: Authorizing `identity:get_user(user_id=7e3563976060470089c252079ed8b72e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:35.313549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856276e-7173-496a-895c-df189fed816e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:35.319844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 154/315] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:35 2022] GET /identity/v3/users/7e3563976060470089c252079ed8b72e => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:35.321412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:35.422150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0c09f-9f8a-43cf-b2fd-c2a2a8a4ef83 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:35.455333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:35.456219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:35.456698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:35.457170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:35.523749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:35.525627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:35.644753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 162/316] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:35.646656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856276e-7173-496a-895c-df189fed816e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:35.747675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856276e-7173-496a-895c-df189fed816e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:35.782731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fe67b-310d-4e72-9d46-87a22a32f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:35.783519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fe67b-310d-4e72-9d46-87a22a32f46a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:35.783801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fe67b-310d-4e72-9d46-87a22a32f46a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:35.784078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fe67b-310d-4e72-9d46-87a22a32f46a None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:35.785301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fe67b-310d-4e72-9d46-87a22a32f46a None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:35.786739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fe67b-310d-4e72-9d46-87a22a32f46a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:35.788761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 155/317] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:35 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:35.794533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:35.895828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d7955-b66b-4cc5-959e-d36f0d3463e2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:35.938621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:35.939649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:35.940125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:35.940609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:35.941680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] RBAC: Adding query filter params (role.id=3b8b637b3f194c399eaf71b58f8f9ae6, scope.project.id=9326b1f4ec284857a398c6df75671714, user.id=7e3563976060470089c252079ed8b72e) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:35.942212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:35.943886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637854-37de-4464-ae3d-00a7adb08fd7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:35.960816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 163/318] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:37:35 2022] GET /identity/v3/role_assignments?user.id=7e3563976060470089c252079ed8b72e&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=3b8b637b3f194c399eaf71b58f8f9ae6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:36.110212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe31e4-9027-43a9-a992-3e42a0beaffc None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.110405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe31e4-9027-43a9-a992-3e42a0beaffc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.110679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe31e4-9027-43a9-a992-3e42a0beaffc None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.112083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 156/319] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:36.118250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae0d71-7622-422d-85d7-55dd1d69073f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.118707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae0d71-7622-422d-85d7-55dd1d69073f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.119089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae0d71-7622-422d-85d7-55dd1d69073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.138227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d2ae0d71-7622-422d-85d7-55dd1d69073f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:36.140011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d2ae0d71-7622-422d-85d7-55dd1d69073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:36.215133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 164/320] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:36.238721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe31e4-9027-43a9-a992-3e42a0beaffc None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.260578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544a14-03d2-4844-96cc-3b20d9111130 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.261254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544a14-03d2-4844-96cc-3b20d9111130 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.261724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99544a14-03d2-4844-96cc-3b20d9111130 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.263553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 165/321] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:36.269429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a8c92-a163-4a53-81e5-8846c699c3d9 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.269714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a8c92-a163-4a53-81e5-8846c699c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.269994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a8c92-a163-4a53-81e5-8846c699c3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.286825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-817a8c92-a163-4a53-81e5-8846c699c3d9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:36.288182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-817a8c92-a163-4a53-81e5-8846c699c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:36.343337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe31e4-9027-43a9-a992-3e42a0beaffc None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.355084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 166/322] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:36.356383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0b3d0-816c-4126-bff8-060c41b13a20 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.356686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0b3d0-816c-4126-bff8-060c41b13a20 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.357108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0b3d0-816c-4126-bff8-060c41b13a20 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.358561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 167/323] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:36.363939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.364457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.364868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.379868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.380687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.380973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.381248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.382399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:36.383429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:36.383829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:36.384590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:36.388329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 157/324] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:36.388329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.450109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 168/325] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:36.451109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.483009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c588ae-e1cd-4f3e-b819-8cec8e27a029 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.514125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.515492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.515780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.516051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.525636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:36.533549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:36.534426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:36.535202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 158/326] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:36.535834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.557134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730fbd1-c61e-42ae-84d6-1b928478e3f3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.589058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.589704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.589979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.590247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] PATH_INFO: `/v3/groups` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.590912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:36.592346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:36.606707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 169/327] 10.222.0.36 () {60 vars in 1134 bytes} [Mon May 16 18:37:36 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:36.607508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.638904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da464f-60f6-42d2-8d3d-68b73221daa2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.674097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.674908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.675186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.675453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.687419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:36.695237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:36.696174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:36.696997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 159/328] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:36.703001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.714605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac54a5-24e3-4fb0-9712-28150ee13f18 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.749070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.749744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.750020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.750296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.751047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:36.751348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:36.752754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:36.761966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 170/329] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:36.768131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.805308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6281-b35a-4a28-b78a-6dc170a43f2e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.838926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.839616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.839899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.840173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.840953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:36.841255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:36.842672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:36.851302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 160/330] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:36.856607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.875355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9bb10-4c59-4279-a068-362e6830b912 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:36.909836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:36.910711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:36.910974 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:36.911250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] PATH_INFO: `/v3/users/3ad58b815f664028937d0ea3884ea30f` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:36.912507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] RBAC: Authorizing `identity:get_user(user_id=3ad58b815f664028937d0ea3884ea30f)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:36.915998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:36.915998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 171/331] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/users/3ad58b815f664028937d0ea3884ea30f => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:36.922525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:36.962710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f8a58-c26a-4c7e-82e6-a94a99ca5996 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:37.001434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:37.002347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:37.002622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:37.002889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] PATH_INFO: `/v3/users/5f96eac557c040c5a47e721aa88cfddc` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:37.004219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] RBAC: Authorizing `identity:get_user(user_id=5f96eac557c040c5a47e721aa88cfddc)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:37.005592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:37.007393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 161/332] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:36 2022] GET /identity/v3/users/5f96eac557c040c5a47e721aa88cfddc => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:37.013756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:37.042286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581668e-5fab-42c9-ada2-d25227e976f8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:37.078745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db71b-a4a1-4804-9f43-e22877823073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:37.079606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db71b-a4a1-4804-9f43-e22877823073 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:37.079884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db71b-a4a1-4804-9f43-e22877823073 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:37.080156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db71b-a4a1-4804-9f43-e22877823073 None admin] PATH_INFO: `/v3/system/users/3ad58b815f664028937d0ea3884ea30f/roles/3b8b637b3f194c399eaf71b58f8f9ae6` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:37.081678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db71b-a4a1-4804-9f43-e22877823073 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3ad58b815f664028937d0ea3884ea30f, role_id=3b8b637b3f194c399eaf71b58f8f9ae6)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:37.083060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db71b-a4a1-4804-9f43-e22877823073 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:37.090180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 172/333] 10.222.0.36 () {58 vars in 1334 bytes} [Mon May 16 18:37:36 2022] PUT /identity/v3/system/users/3ad58b815f664028937d0ea3884ea30f/roles/3b8b637b3f194c399eaf71b58f8f9ae6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:37.122437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822e02-8b50-4eff-ad5f-65300fed4a22 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:37.161437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:37.162027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:37.162309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:37.162582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:37.163611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] RBAC: Adding query filter params (role.id=c1095be460574bfbb5445ac9e464a9b7, user.id=5f96eac557c040c5a47e721aa88cfddc, scope.system=all) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:37.163927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:37.165449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75cded-4ea2-42ab-9836-642755489839 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:37.179915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 162/334] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 16 18:37:37 2022] GET /identity/v3/role_assignments?user.id=5f96eac557c040c5a47e721aa88cfddc&scope.system=all&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:39.102472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478f5d1-9374-4e5e-86e0-0bde80c78fc5 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:39.103374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478f5d1-9374-4e5e-86e0-0bde80c78fc5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:39.103811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478f5d1-9374-4e5e-86e0-0bde80c78fc5 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:39.105605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 173/335] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:39.111235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:39.111994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:39.112415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:39.130776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:39.132178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:39.206408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 163/336] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:39.233791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478f5d1-9374-4e5e-86e0-0bde80c78fc5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:39.339362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478f5d1-9374-4e5e-86e0-0bde80c78fc5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:39.375476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:39.376269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:39.376551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:39.376859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:39.387524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:39.395283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:39.396200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:39.397049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:39.398826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 174/337] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:39.402735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:39.504618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bac32-968b-41ff-8c62-99bd20b6461c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:39.540098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:39.540813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:39.541093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:39.541365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:39.542117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:39.542420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:39.543788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:39.554216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 164/338] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:39.558798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:39.660775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab71a-f7c6-428d-8e67-64cbe82a3a37 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:39.695811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:39.696462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:39.696772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:39.697044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:39.766128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:39.767549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:39.867768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 175/339] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:39.873534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:39.987035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928f1c7-6744-4213-bace-20e4b9cb982e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:40.020431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:40.021235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.021514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.021789 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.022975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:40.024414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:40.026482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 165/340] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:39 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:40.032844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:40.136026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627572-dc2b-4813-aafc-8cce71f9e400 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:40.171445 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0c96d-4652-4e64-ab96-0ea4453b038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:40.172300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0c96d-4652-4e64-ab96-0ea4453b038c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.172578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0c96d-4652-4e64-ab96-0ea4453b038c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.172883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0c96d-4652-4e64-ab96-0ea4453b038c None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.178949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0c96d-4652-4e64-ab96-0ea4453b038c None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:40.180387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0c96d-4652-4e64-ab96-0ea4453b038c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:40.186878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 176/341] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:40 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:40.188199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:40.299675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf0c1-1a07-42f1-8a7b-29c306221405 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:40.339110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:40.339780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.340060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.340334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.341266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=d32e19faf0104e77a963e4c18b759888) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:40.341580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:40.343145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e854f0-dc09-4b50-a4e8-73c48b75c895 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:40.351029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 166/342] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:40 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:40.626161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f46d9-e840-4dd8-a99e-03bcdd98de01 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.626821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f46d9-e840-4dd8-a99e-03bcdd98de01 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.627324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f46d9-e840-4dd8-a99e-03bcdd98de01 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.629211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 177/343] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:40.632813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd09c-3435-4d69-b417-4b1ce53cc065 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.633343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd09c-3435-4d69-b417-4b1ce53cc065 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.633816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd09c-3435-4d69-b417-4b1ce53cc065 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.633918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ccd8-d9f1-4e34-a226-f3235ff5ac34 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.634145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ccd8-d9f1-4e34-a226-f3235ff5ac34 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.634657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ccd8-d9f1-4e34-a226-f3235ff5ac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.635716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 167/344] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:40.642790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.643076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.643358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.656858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-26e6ccd8-d9f1-4e34-a226-f3235ff5ac34 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:40.658601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-26e6ccd8-d9f1-4e34-a226-f3235ff5ac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:40.672701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:40.674101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:40.744425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 178/345] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:40.746121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1c084-4de5-4a4d-b8e3-03388da1cbf7 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.746620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1c084-4de5-4a4d-b8e3-03388da1cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.747107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1c084-4de5-4a4d-b8e3-03388da1cbf7 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.749435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 179/346] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:40.752882 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 168/347] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 16 18:37:40.754754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.755249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.755811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.776868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:40.782959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:40.782959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:40.877887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346bd6-9e68-4cd2-a200-c9580cc519d6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:40.921068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:40.922191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:40.922668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:40.923127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:40.934364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:40.937721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 180/348] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:40.938581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:40.946319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:40.947851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 16 18:37:40.949050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:40.959767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 169/349] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:40.962852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.065007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c798f59-ae8d-4dfa-90b3-eddd523cb62c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.070783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cc951-4193-4bd9-8ba8-db0aeedece82 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.102009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.102681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.102968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.103241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.104545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:41.104545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:41.105915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:41.116913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 170/350] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:41.117240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5eae8-f271-4502-afe6-62aaa117dc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.118056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5eae8-f271-4502-afe6-62aaa117dc0c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.118336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5eae8-f271-4502-afe6-62aaa117dc0c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.118607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5eae8-f271-4502-afe6-62aaa117dc0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.118924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.129814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5eae8-f271-4502-afe6-62aaa117dc0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-f5a5eae8-f271-4502-afe6-62aaa117dc0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:41.138952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:41.140297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 181/351] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:41.141828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa27584-e3aa-4452-81ea-e8b1e44312b0 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.142549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa27584-e3aa-4452-81ea-e8b1e44312b0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.142549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa27584-e3aa-4452-81ea-e8b1e44312b0 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.143343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 182/352] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:41.143864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa27584-e3aa-4452-81ea-e8b1e44312b0 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.238597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ce6eb-0eb3-4132-bc8b-32b5aff140cc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.255318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa27584-e3aa-4452-81ea-e8b1e44312b0 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.274197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.274994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.275275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.275547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.286723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:41.294801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:41.296329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 171/353] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:41.297621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.302529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f6247-53bf-498b-8df1-afbc66ef188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.303413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f6247-53bf-498b-8df1-afbc66ef188e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.303695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f6247-53bf-498b-8df1-afbc66ef188e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.303971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f6247-53bf-498b-8df1-afbc66ef188e None admin] PATH_INFO: `/v3/users/3ad58b815f664028937d0ea3884ea30f` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.305265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f6247-53bf-498b-8df1-afbc66ef188e None admin] RBAC: Authorizing `identity:get_user(user_id=3ad58b815f664028937d0ea3884ea30f)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:41.306725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f6247-53bf-498b-8df1-afbc66ef188e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:41.308679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 183/354] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/users/3ad58b815f664028937d0ea3884ea30f => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:41.309861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.310149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.310422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.327226 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:41.328422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:41.399728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07ec6-25de-4bdc-979f-da4851cae837 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.400722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 184/355] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:41.401548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.434225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.434892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.435171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.435445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.436225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:41.436531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:41.439146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:41.449005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 172/356] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:41.449772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.526118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def5cc8-b0c0-449a-9ce8-1e23c93907c5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.561752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e58b-0880-4b3c-8f54-a878a1c447a9 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.562951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.563624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.563901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.564170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.564971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:41.565274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:41.566639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:41.580094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 185/357] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:41.580094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.605627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.606634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.607118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.607585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.608682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] RBAC: Adding query filter params (role.id=3b8b637b3f194c399eaf71b58f8f9ae6, user.id=3ad58b815f664028937d0ea3884ea30f, scope.system=all) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:41.609217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:41.610984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:41.625983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 173/358] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/role_assignments?user.id=3ad58b815f664028937d0ea3884ea30f&scope.system=all&role.id=3b8b637b3f194c399eaf71b58f8f9ae6 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:41.627192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.685389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade32d4-eeab-4cc6-890d-1ddfd5c8ab18 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.723372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.724041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.724346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.724593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.800679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:41.800679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:41.837983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3821cd-bf49-4863-b06f-d5fedcbc6464 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:41.894548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab138c3-cc68-413f-b624-b4af8a0c5a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:41.895373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab138c3-cc68-413f-b624-b4af8a0c5a51 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.895653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab138c3-cc68-413f-b624-b4af8a0c5a51 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.895928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab138c3-cc68-413f-b624-b4af8a0c5a51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.912837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 186/359] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:41.913696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:41.918435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab138c3-cc68-413f-b624-b4af8a0c5a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-dab138c3-cc68-413f-b624-b4af8a0c5a51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:41.927893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:41.929070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:41.930049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:41.933240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 174/360] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:41.937917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a2c1-7096-4490-a900-bbccd2a9c56e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:41.937917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a2c1-7096-4490-a900-bbccd2a9c56e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:41.937917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05a2c1-7096-4490-a900-bbccd2a9c56e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:41.939555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 175/361] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:41.940249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a2c1-7096-4490-a900-bbccd2a9c56e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.055251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca5ead-1bc8-4584-96eb-f29c3c1bad81 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.082535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05a2c1-7096-4490-a900-bbccd2a9c56e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.101516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a463aa2-d299-46e0-a39d-9201c84d7db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.102168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a463aa2-d299-46e0-a39d-9201c84d7db3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.102449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a463aa2-d299-46e0-a39d-9201c84d7db3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.102718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a463aa2-d299-46e0-a39d-9201c84d7db3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.129422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.130293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.130574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.130846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.133458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:42.134909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:42.136618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 176/362] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:42.137401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.196744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a463aa2-d299-46e0-a39d-9201c84d7db3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:42.198167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a463aa2-d299-46e0-a39d-9201c84d7db3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:42.251862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889f61e-b960-4a4a-b133-ca83334eb909 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.307615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.308450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.308764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.309047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.309794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:42.310099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:42.311999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:42.313537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 187/363] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:42.315020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.315309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.315584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.330141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 177/364] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:42.331302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.349068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:42.350264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:42.431668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 188/365] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:42.432513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.461829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacf621-856b-4edd-9c89-5c7ec8821cab None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.510894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.511764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.512047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.512323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.513663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:42.515130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:42.517113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 178/366] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:42.520756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.566949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e3566-0425-4a87-a956-27834da3c7b4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.607457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.608965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.609244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.609508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.610897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:42.612361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:42.614275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 189/367] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:42.614876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.627359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745dc2b-8b0e-4cdf-8360-ffcbc726661b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.683516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.684180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.684460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.684764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.746836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450d37d-eda8-4d53-b1e4-f57be391af28 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:42.772058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:42.773546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:42.790482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:42.791272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:42.791549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:42.791819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:42.807134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:42.814999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:42.815873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:42.816721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 190/368] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:42.817472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.916007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 179/369] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:42.917144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:42.953585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab53960-0e47-4800-beff-969d503f69c0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.000487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.001634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.002458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.002736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.003582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=d32e19faf0104e77a963e4c18b759888) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:43.004038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.008132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.017842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 191/370] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=d584b6d85205413eb17765aa9c683df4 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.019031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.045663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6317a5-bb21-4391-becf-a02617c14957 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.093603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.094473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.094750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.095023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.096860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.098307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.104777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 180/371] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:42 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.104777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.160146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2e739-3744-4557-b7f3-48bbacf712b2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.197207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.198929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.199424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.199914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.200928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:43.201996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.203639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f5145-135f-4d60-a058-7d2f379e24e1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.213440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 192/372] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.220127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58448f8-6193-4af4-aa9f-1f68242443cf None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.220127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58448f8-6193-4af4-aa9f-1f68242443cf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.220127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58448f8-6193-4af4-aa9f-1f68242443cf None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.220407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3656ab-aaca-4921-8816-715c0854e9a7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.221107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 193/373] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:43.221877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58448f8-6193-4af4-aa9f-1f68242443cf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.255300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.256092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.256374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.256732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.257919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.259376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.261615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 181/374] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.262245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.339875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58448f8-6193-4af4-aa9f-1f68242443cf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.382601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.383372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.383722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.384067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.385058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=9326b1f4ec284857a398c6df75671714) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:43.385441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.386899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44f2c-f965-4a1c-b8d3-a552d29f8591 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.393963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 194/375] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.395647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.396521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.397360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.399221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b359521-0460-43db-8969-69315be58ae8 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.419631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:43.422277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:43.441847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.442395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.442770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.443175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.520850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 195/376] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:43.521673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.544064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.546220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.660621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fd00c-b56d-4d75-8e76-2c28d95f0e3d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.678879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 182/377] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:43.679773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.697632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba75170-9005-4419-b890-860872745b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.698528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba75170-9005-4419-b890-860872745b81 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.698813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba75170-9005-4419-b890-860872745b81 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.699087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba75170-9005-4419-b890-860872745b81 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.700925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba75170-9005-4419-b890-860872745b81 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.702942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba75170-9005-4419-b890-860872745b81 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.710475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 196/378] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.710475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba75170-9005-4419-b890-860872745b81 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.793636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c96aa-da65-445f-87a9-36aa51249c38 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.806989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba75170-9005-4419-b890-860872745b81 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.833302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.834460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.834936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.835404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.842849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.843708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.843989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.844317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] PATH_INFO: `/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.845626 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] RBAC: Authorizing `identity:get_project(project_id=b27a519f15bc4f1e98fc811594ab8cdd)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.847144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:43.849226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:43.852838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 197/379] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:43.853663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:43.858082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:43.858954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:43.859732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:43.862114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 183/380] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:43.865905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:43.959173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf906473-1326-4eaf-93e8-3ddb65cb8135 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.981737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af382926-e25a-4d48-a1c2-348fdffb4806 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:43.995133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:43.995838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:43.996117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:43.996439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:43.997301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=9326b1f4ec284857a398c6df75671714) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:43.997609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:43.999580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.005604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 198/381] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=d584b6d85205413eb17765aa9c683df4 => generated 260 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:44.012669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:44.036497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:44.037396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:44.037674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:44.037946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] PATH_INFO: `/v3/groups/c6191926cc194e2f8efaaad68c52701f` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:44.045311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] RBAC: Authorizing `identity:get_group(group_id=c6191926cc194e2f8efaaad68c52701f)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:44.046695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.048371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 184/382] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:43 2022] GET /identity/v3/groups/c6191926cc194e2f8efaaad68c52701f => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:44.054252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:44.116594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d43b2-c81d-40c1-b943-3a2182bcf3a8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:44.153160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:44.153840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:44.154119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:44.154455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:44.155226 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:44.155528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:44.157006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.164541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd330b-3839-4c2d-89c8-4fea879ab39b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:44.166908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 199/383] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:44.172404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:44.200808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:44.202104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:44.202384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:44.202655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:44.203477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] RBAC: Adding query filter params (group.id=c6191926cc194e2f8efaaad68c52701f, role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=b27a519f15bc4f1e98fc811594ab8cdd) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:44.203783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:44.205251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.211778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 185/384] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:44 2022] GET /identity/v3/role_assignments?group.id=c6191926cc194e2f8efaaad68c52701f&scope.project.id=b27a519f15bc4f1e98fc811594ab8cdd&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:44.333810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224d8aa-6f0a-4786-ada1-ff3a76acfcf3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:44.382388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:44.383278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:44.383559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:44.383835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:44.385189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:44.386692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.392860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 200/385] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:44 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:44.397810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:44.514041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6039c-efc2-436f-8ff0-c65b40fcaea3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:44.561921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f600b0-8a87-4197-9716-ccdaac520ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:44.562784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f600b0-8a87-4197-9716-ccdaac520ec2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:44.563058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f600b0-8a87-4197-9716-ccdaac520ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:44.563327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f600b0-8a87-4197-9716-ccdaac520ec2 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:44.569389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f600b0-8a87-4197-9716-ccdaac520ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:44.570833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f600b0-8a87-4197-9716-ccdaac520ec2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.572761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 186/386] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:44 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:44.578751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:44.696900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61df22-22d1-4003-8e7c-211602abe8e9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:44.733596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1512384-1d03-4ac0-bdb9-ba0f62c4fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:44.734718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512384-1d03-4ac0-bdb9-ba0f62c4fbbe None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:44.735145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512384-1d03-4ac0-bdb9-ba0f62c4fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:44.735559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512384-1d03-4ac0-bdb9-ba0f62c4fbbe None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888/groups/bd534386a5d04ee499955469b24f2a27/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:44.738082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1512384-1d03-4ac0-bdb9-ba0f62c4fbbe None admin] RBAC: Authorizing `identity:create_grant(project_id=d32e19faf0104e77a963e4c18b759888, group_id=bd534386a5d04ee499955469b24f2a27, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:44.739694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1512384-1d03-4ac0-bdb9-ba0f62c4fbbe None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:44.759556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 201/387] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 16 18:37:44 2022] PUT /identity/v3/projects/d32e19faf0104e77a963e4c18b759888/groups/bd534386a5d04ee499955469b24f2a27/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:45.667859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95651cff-5978-430a-a002-c81b9661c57e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:45.668413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95651cff-5978-430a-a002-c81b9661c57e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:45.668413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95651cff-5978-430a-a002-c81b9661c57e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:45.669924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 187/388] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:45.675192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:45.675942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:45.676366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:45.702235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:45.703643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:45.770829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 202/389] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:45.804962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95651cff-5978-430a-a002-c81b9661c57e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:45.930651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95651cff-5978-430a-a002-c81b9661c57e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:45.976092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:45.976931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:45.977217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:45.977490 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:45.988368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:46.002070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:46.002938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:46.003726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 188/390] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:46.009024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:46.173355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32500b-b236-4239-81f8-6aab72ffe847 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:46.223229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:46.224257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.224790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.225262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.226240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:46.226742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:46.228341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:46.238095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 203/391] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:46.245912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:46.380900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0674-9d65-43fa-bebc-6a3eea67c08f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:46.420668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601afd06-1324-437c-8dad-2c153815e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:46.421549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afd06-1324-437c-8dad-2c153815e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.421834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afd06-1324-437c-8dad-2c153815e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.422103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afd06-1324-437c-8dad-2c153815e8c9 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.423287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601afd06-1324-437c-8dad-2c153815e8c9 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:46.424767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601afd06-1324-437c-8dad-2c153815e8c9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:46.426555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 189/392] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:46 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:46.431852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:46.553270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570b0b9-ff0f-473c-b685-7d914c9773fc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:46.564045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b2531-6b7d-4427-a930-571a5517f40a None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.564343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b2531-6b7d-4427-a930-571a5517f40a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.564746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b2531-6b7d-4427-a930-571a5517f40a None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.565995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 190/393] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:37:46.578029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.578314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.578595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.586860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:46.587921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.588435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.588913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.590247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:46.592363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:46.594553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 204/394] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:46 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:46.601492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:46.609635 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:46.610827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:46.706004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbcf76-b9d5-4e3b-b01e-a68536ae8cb9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:46.740533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:46.741963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.742458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.742916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888/groups/bd534386a5d04ee499955469b24f2a27/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.743162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 191/395] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:46.745911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d32e19faf0104e77a963e4c18b759888, group_id=bd534386a5d04ee499955469b24f2a27, role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:46.747595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:46.761074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 205/396] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 16 18:37:46 2022] PUT /identity/v3/projects/d32e19faf0104e77a963e4c18b759888/groups/bd534386a5d04ee499955469b24f2a27/roles/d584b6d85205413eb17765aa9c683df4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:46.774696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:46.919474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a485065-8c32-4c17-8b0d-a9f7304fb390 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:46.974708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61faa36-5c0f-4f9c-839a-131af9903e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:46.975520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61faa36-5c0f-4f9c-839a-131af9903e3c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:46.975802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61faa36-5c0f-4f9c-839a-131af9903e3c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:46.976075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61faa36-5c0f-4f9c-839a-131af9903e3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:46.996826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61faa36-5c0f-4f9c-839a-131af9903e3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a61faa36-5c0f-4f9c-839a-131af9903e3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:47.005409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:47.006310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:47.007106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:47.009493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 192/397] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:47.016331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.056568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420fa9e-6628-4ed9-9dc0-a3512b17f6af None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.056899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420fa9e-6628-4ed9-9dc0-a3512b17f6af None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.057180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420fa9e-6628-4ed9-9dc0-a3512b17f6af None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.062373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 193/398] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:47.063923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.064211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.064485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.083456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:47.084646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:47.136576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2190-f30e-4088-a79d-b26874f446e6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.160770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 194/399] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:47.174380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.174968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.175247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.175517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.176258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:47.176746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:47.177367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.177952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258e95e-cf44-43c1-b3f8-a5ca5d183498 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:47.193199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 206/400] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:47.194615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a462a-5042-4bed-b64e-5e4d079ef69c None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.194901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a462a-5042-4bed-b64e-5e4d079ef69c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.195172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a462a-5042-4bed-b64e-5e4d079ef69c None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.196449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 207/401] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:47.202096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a462a-5042-4bed-b64e-5e4d079ef69c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.284466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72653142-befa-45b0-b8b7-09d29b07adc8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.319886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c657b-c4a8-4de2-93ef-406e97b9cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.320888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c657b-c4a8-4de2-93ef-406e97b9cc7f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.321167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c657b-c4a8-4de2-93ef-406e97b9cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.321437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c657b-c4a8-4de2-93ef-406e97b9cc7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.323332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a462a-5042-4bed-b64e-5e4d079ef69c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.333448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c657b-c4a8-4de2-93ef-406e97b9cc7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f82c657b-c4a8-4de2-93ef-406e97b9cc7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:47.345179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:47.346135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:47.347004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:47.352742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 195/402] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:47.354572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.355030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.355440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.367419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.368522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.368925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.369327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.371575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:47.373325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:47.375284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:47.375599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 208/403] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:47.376493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.381951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:47.468123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 196/404] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:47.469036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.508311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c115ee5-2981-4c0d-bb42-8cf4333435f7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.542842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.543959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.544404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.544710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.545489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:47.545840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:47.547236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:47.557121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 209/405] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:47.557121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.589296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097f923-8ee0-4f88-ab28-c1579df91640 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.621772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d47afd-58e8-45b8-89e8-96746f7ba61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.622621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47afd-58e8-45b8-89e8-96746f7ba61a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.622902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47afd-58e8-45b8-89e8-96746f7ba61a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.623204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47afd-58e8-45b8-89e8-96746f7ba61a None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.624375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d47afd-58e8-45b8-89e8-96746f7ba61a None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:47.625841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d47afd-58e8-45b8-89e8-96746f7ba61a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:47.627515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 197/406] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:47.633307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a9a39-4f22-4785-a841-13a76fb4a266 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.633591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a9a39-4f22-4785-a841-13a76fb4a266 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.633865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a9a39-4f22-4785-a841-13a76fb4a266 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.636786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 198/407] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:47.636786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a9a39-4f22-4785-a841-13a76fb4a266 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.653972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8c62-5ff9-41c4-8881-6da7ed121c26 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.689555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.690238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.690557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.690782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.704561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:47.712877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:47.714182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:47.714985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:47.717479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 210/408] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:47.718557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.745031 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a9a39-4f22-4785-a841-13a76fb4a266 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.781357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c52b-2d44-40a9-95c1-9869598810b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.782942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c52b-2d44-40a9-95c1-9869598810b3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.783229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c52b-2d44-40a9-95c1-9869598810b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.783506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c52b-2d44-40a9-95c1-9869598810b3 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.785127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c52b-2d44-40a9-95c1-9869598810b3 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:47.787311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c52b-2d44-40a9-95c1-9869598810b3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:47.793353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 199/409] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:47.793353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.793353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.793353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.808915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:47.810160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:47.815276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b0480-ae97-4aae-8abf-6172928ecae3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:47.847550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:47.848421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:47.848733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:47.849011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714/groups/bd534386a5d04ee499955469b24f2a27/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:47.850963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] RBAC: Authorizing `identity:create_grant(project_id=9326b1f4ec284857a398c6df75671714, group_id=bd534386a5d04ee499955469b24f2a27, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:47.852405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:47.866212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 211/410] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 16 18:37:47 2022] PUT /identity/v3/projects/9326b1f4ec284857a398c6df75671714/groups/bd534386a5d04ee499955469b24f2a27/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:47.876962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.882223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 200/411] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:47.883334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:47.989873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dbfaf-ff03-42dc-94bf-2c9af907c743 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.006255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cae27-b910-4f3c-b453-2036a250f692 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.028879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.029728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.030012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.030290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.031264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:48.031575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.033351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.042917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 212/412] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:48.043973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.044464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.045444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.045725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.045996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.047248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.048761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.055753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 201/413] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:47 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:48.056588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.152304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d4cc-df34-4949-b93f-af03997025ee None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.177289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86679084-2bfa-4bc7-9130-9bb86e384e89 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.193722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.194902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.195462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.195741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.208296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:48.216934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:48.217623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:48.218461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:48.219230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:48.219808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.219808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.219808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] PATH_INFO: `/v3/groups/c6191926cc194e2f8efaaad68c52701f` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.219972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 213/414] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:48.219972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.221492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] RBAC: Authorizing `identity:get_group(group_id=c6191926cc194e2f8efaaad68c52701f)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.222957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.225900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 202/415] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:48 2022] GET /identity/v3/groups/c6191926cc194e2f8efaaad68c52701f => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:48.226678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.331228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c75d3-d381-479b-8bf8-4e82ee55a8dd None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.344931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c221da-c480-4bde-839e-2b040411b2c6 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.365187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.366224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.366610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.366988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714/groups/bd534386a5d04ee499955469b24f2a27/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.369358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9326b1f4ec284857a398c6df75671714, group_id=bd534386a5d04ee499955469b24f2a27, role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.370951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.386978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 214/416] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 16 18:37:48 2022] PUT /identity/v3/projects/9326b1f4ec284857a398c6df75671714/groups/bd534386a5d04ee499955469b24f2a27/roles/d584b6d85205413eb17765aa9c683df4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:48.387267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.387863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.389346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.389628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.389906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.390680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:48.390986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.392390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.414521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 203/417] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:48.424380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.574393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f6477-16c6-48d2-a443-0a21287458e0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.574620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b4e02-1e33-45f5-bafc-d5c3d073e4b4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.617589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b181d3-873e-486c-a034-f2225512484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.618583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b181d3-873e-486c-a034-f2225512484d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.619057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b181d3-873e-486c-a034-f2225512484d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.619542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b181d3-873e-486c-a034-f2225512484d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.622180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.623595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.623595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.623762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] PATH_INFO: `/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.625540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=b27a519f15bc4f1e98fc811594ab8cdd)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.626609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.628826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 215/418] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:48 2022] GET /identity/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:48.634665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.718898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b181d3-873e-486c-a034-f2225512484d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.720371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b181d3-873e-486c-a034-f2225512484d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.752500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13665a5-ba00-4e91-b2ec-7e0753bfc9d4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:48.786595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:48.787480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:48.787761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:48.788035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] PATH_INFO: `/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd/groups/c6191926cc194e2f8efaaad68c52701f/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:48.790058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] RBAC: Authorizing `identity:create_grant(project_id=b27a519f15bc4f1e98fc811594ab8cdd, group_id=c6191926cc194e2f8efaaad68c52701f, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:48.791512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:48.807889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 216/419] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 16 18:37:48 2022] PUT /identity/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd/groups/c6191926cc194e2f8efaaad68c52701f/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:37:48.823770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 204/420] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:48.831391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:48.964348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c35b0-c927-4396-a9a8-42095c6332ff None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:49.000533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:49.001437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.001715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.001988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.003207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:49.004674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:49.006699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 217/421] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:48 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:49.012218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b181d3-873e-486c-a034-f2225512484d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:49.154940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b181d3-873e-486c-a034-f2225512484d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:49.192387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f55a6-fd2b-4156-8961-b478a8646dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:49.193260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f55a6-fd2b-4156-8961-b478a8646dcc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.193543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f55a6-fd2b-4156-8961-b478a8646dcc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.193820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f55a6-fd2b-4156-8961-b478a8646dcc None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.194995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f55a6-fd2b-4156-8961-b478a8646dcc None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:49.196416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f55a6-fd2b-4156-8961-b478a8646dcc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:49.198070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 205/422] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:49 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:49.204461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:49.316168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ca00-cc6b-41f3-ac00-ca51d102461f None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.316617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ca00-cc6b-41f3-ac00-ca51d102461f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.317020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ca00-cc6b-41f3-ac00-ca51d102461f None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.318738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 206/423] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:37:49.324248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.324619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.325016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.340935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bdba1-8615-4995-bdd8-5cb5ba0d8976 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:49.360536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:49.362123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:49.403944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:49.404901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.405319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.405730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.406696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=d32e19faf0104e77a963e4c18b759888) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:49.407174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:49.409024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:49.416426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 218/424] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:49 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:49.502734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 207/425] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:49.550558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:49.690667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e13b3-45d8-4660-968e-05b46aa585b1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:49.729442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:49.730485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.730906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.731340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.746393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:49.755242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:49.756727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:49.757702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:49.764950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 219/426] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:49.765933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:49.891771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926d2ef-71ef-4cc4-838a-c399970abb14 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:49.933781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:49.934461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:49.934744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:49.935018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:49.935821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:49.936128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:49.937532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:49.946206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 208/427] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:49.952372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:50.080105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8f220-f982-4fe0-ba03-f997f4b7d528 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:50.120685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458133f-93f5-432d-9963-e41a742862f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:50.121684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458133f-93f5-432d-9963-e41a742862f7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.122199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458133f-93f5-432d-9963-e41a742862f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.122680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458133f-93f5-432d-9963-e41a742862f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.198654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458133f-93f5-432d-9963-e41a742862f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:50.200490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458133f-93f5-432d-9963-e41a742862f7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:50.304066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 220/428] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:50.313724 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:50.400749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081bc57-be70-458a-9820-1573068cf9a7 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.401373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081bc57-be70-458a-9820-1573068cf9a7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.401845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081bc57-be70-458a-9820-1573068cf9a7 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.403619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 221/429] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:50.409524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.410113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.410646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.432976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:50.434477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:50.453547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5cafe-b415-4e22-9e3f-8fbbfbfd19cb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:50.487409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:50.488412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.488713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.488987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] PATH_INFO: `/v3/projects/d32e19faf0104e77a963e4c18b759888` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.490376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] RBAC: Authorizing `identity:get_project(project_id=d32e19faf0104e77a963e4c18b759888)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:50.491819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:50.494134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 209/430] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:50 2022] GET /identity/v3/projects/d32e19faf0104e77a963e4c18b759888 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:50.499797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:50.510007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 222/431] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:50.538663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:50.620274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757a1b-f2b1-4b6e-96e2-a0788f4d08cf None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:50.646638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90f870-01b4-4f9a-b774-0d957010a304 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:50.658403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:50.659513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.659994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.660488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.661884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:50.663638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:50.665789 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 210/432] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:50 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:50.672320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:50.687529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:50.688707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.689181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.689658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.700332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:50.708327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:50.709380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 16 18:37:50.710167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:50.712416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 223/433] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:50.718349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:50.772612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207642d9-6571-4da6-9560-869c4ae3f11a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:50.808183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:50.809156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.809654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.810097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=d32e19faf0104e77a963e4c18b759888) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:50.811615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:50.813225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:50.824672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 211/434] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:50 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=d32e19faf0104e77a963e4c18b759888&role.id=d584b6d85205413eb17765aa9c683df4 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:50.824860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca204ff9-55f3-4183-8fee-0c67cc239b71 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:50.859103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:50.859940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:50.860224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:50.860498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:50.861285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] RBAC: Adding query filter params (name=member) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:50.861592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:50.862994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef07c5d-9692-473e-b614-9ec06a1c270b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:50.872517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 224/435] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:50.878163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.002253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2482a-fd0c-4677-88d9-e89efbc5bd2e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.041833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.042849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.043130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.043399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.112647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:51.114101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:51.192113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70250c70-4e2f-447a-823a-f3c2d96df2f2 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.192396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70250c70-4e2f-447a-823a-f3c2d96df2f2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.192722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70250c70-4e2f-447a-823a-f3c2d96df2f2 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.194201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 225/436] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:51.199197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.199484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.199761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.215388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 212/437] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:51.219819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:51.221199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:51.222016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.315570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 226/438] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:51.338973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.345595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a90ab-7eb5-4eaf-93d1-53030e494dfd None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.382083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.382940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.383220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.383492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.384746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:51.386198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:51.392967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 213/439] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:51.393672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.451618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8530466-a10e-4b7a-9afb-211336c3c2f1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.483696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bb3a7-172e-4c70-bc6d-23be844ac913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.484453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bb3a7-172e-4c70-bc6d-23be844ac913 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.484740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bb3a7-172e-4c70-bc6d-23be844ac913 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.485017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bb3a7-172e-4c70-bc6d-23be844ac913 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.486453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47b0a7-58fe-48dd-a48f-25c60a46c4c3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.494593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bb3a7-172e-4c70-bc6d-23be844ac913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-107bb3a7-172e-4c70-bc6d-23be844ac913 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:51.503139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:51.504230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 227/440] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:51.505982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aac92-4d27-43a8-afb9-cd57477484b8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.506688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aac92-4d27-43a8-afb9-cd57477484b8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.506688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aac92-4d27-43a8-afb9-cd57477484b8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.507794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 228/441] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:51.510190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aac92-4d27-43a8-afb9-cd57477484b8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.520559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9879-61ff-4a6e-9aa8-d313e37a123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.521432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9879-61ff-4a6e-9aa8-d313e37a123f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.521714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9879-61ff-4a6e-9aa8-d313e37a123f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.521984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9879-61ff-4a6e-9aa8-d313e37a123f None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.523147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9879-61ff-4a6e-9aa8-d313e37a123f None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:51.524566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9879-61ff-4a6e-9aa8-d313e37a123f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:51.526225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 214/442] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:51.527357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.527663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.527966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.543502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:51.544682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:51.612341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 215/443] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:51.613258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.629887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aac92-4d27-43a8-afb9-cd57477484b8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.665156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.665823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.666151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.666667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.667507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:51.667809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:51.669515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:51.679526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 229/444] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:51.680327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.726352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aaa8e-8270-4def-afc9-63e8ddab6bc6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.761057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.761935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.762210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.762480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.763298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=c1095be460574bfbb5445ac9e464a9b7, scope.project.id=9326b1f4ec284857a398c6df75671714) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:51.763661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:51.765591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:51.771162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 216/445] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=c1095be460574bfbb5445ac9e464a9b7 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:51.771926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.782322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61117f6b-a99f-4d2e-a3c2-4287bda29085 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.819962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.820845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.821126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.821403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.833836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:51.841794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:51.842795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:51.843570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 230/446] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:51.850980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:51.885545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eab4ae-fce5-416b-82e4-22e04642726e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:51.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:51.922819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:51.923102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:51.923380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:51.966707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904030da-6639-4960-89a1-1840db8b1e81 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.000593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.001667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.001941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.002215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.002956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:52.003257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.005093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.006295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.007731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.014831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 231/447] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:52.018554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.120291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7de7f-675c-4853-ab1f-cf0b81339847 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.124293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 217/448] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:37:52.129119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.155026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.155677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.156927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.156927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.221733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.223186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.225631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a2e54-3745-4885-83ef-44daaf20751e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.257603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.258433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.258711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.258980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] PATH_INFO: `/v3/projects/9326b1f4ec284857a398c6df75671714` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.260153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] RBAC: Authorizing `identity:get_project(project_id=9326b1f4ec284857a398c6df75671714)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.261605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.263263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 218/449] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/projects/9326b1f4ec284857a398c6df75671714 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:52.268690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.316143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 232/450] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 18:37:52.321594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.359918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb83100-d4d9-41dd-80fb-22f8168ed703 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.390486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.391302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.391574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.391841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] PATH_INFO: `/v3/groups/bd534386a5d04ee499955469b24f2a27` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.393016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] RBAC: Authorizing `identity:get_group(group_id=bd534386a5d04ee499955469b24f2a27)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.394427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.395992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 219/451] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/groups/bd534386a5d04ee499955469b24f2a27 => generated 226 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:52.401452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.412841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12fac9-e5a4-4137-a4ef-f4c0509c0f1d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.444551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.445410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.445692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.445968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] PATH_INFO: `/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.447141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] RBAC: Authorizing `identity:get_project(project_id=b27a519f15bc4f1e98fc811594ab8cdd)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.448567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.450403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 233/452] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/projects/b27a519f15bc4f1e98fc811594ab8cdd => generated 344 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:52.455435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.492856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe777fc-3701-4fc4-ab6b-1fa2bd642be4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.523610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.524179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.524456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.524763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.525575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] RBAC: Adding query filter params (group.id=bd534386a5d04ee499955469b24f2a27, role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=9326b1f4ec284857a398c6df75671714) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:52.525880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.527283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.533529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 220/453] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/role_assignments?group.id=bd534386a5d04ee499955469b24f2a27&scope.project.id=9326b1f4ec284857a398c6df75671714&role.id=d584b6d85205413eb17765aa9c683df4 => generated 611 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:52.548303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a386f-4705-4773-b023-866fee7927c8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.583002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad6802-1175-4d0e-a87f-74c2be4c6d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.583864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6802-1175-4d0e-a87f-74c2be4c6d07 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.584143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6802-1175-4d0e-a87f-74c2be4c6d07 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.584413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad6802-1175-4d0e-a87f-74c2be4c6d07 None admin] PATH_INFO: `/v3/groups/c6191926cc194e2f8efaaad68c52701f` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.585703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad6802-1175-4d0e-a87f-74c2be4c6d07 None admin] RBAC: Authorizing `identity:get_group(group_id=c6191926cc194e2f8efaaad68c52701f)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.587203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad6802-1175-4d0e-a87f-74c2be4c6d07 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.588979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 234/454] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/groups/c6191926cc194e2f8efaaad68c52701f => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:37:52.595714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:52.696229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0447e82-035d-4e2d-b55b-2075933bc2eb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:52.729472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:52.730118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:52.730401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:52.730676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:52.731505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] RBAC: Adding query filter params (group.id=c6191926cc194e2f8efaaad68c52701f, role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=b27a519f15bc4f1e98fc811594ab8cdd) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:52.731813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:52.733303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81768f1-f371-4a7e-88ee-45742c6f04ac None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:52.739026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 221/455] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 16 18:37:52 2022] GET /identity/v3/role_assignments?group.id=c6191926cc194e2f8efaaad68c52701f&scope.project.id=b27a519f15bc4f1e98fc811594ab8cdd&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:55.472885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b6c88-7cb9-4515-aecf-7d87540a3eda None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.473900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b6c88-7cb9-4515-aecf-7d87540a3eda None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.474483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b6c88-7cb9-4515-aecf-7d87540a3eda None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.476026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 235/456] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:55.486531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5e26-c8e5-4952-b19e-3826eb639b58 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.487372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5e26-c8e5-4952-b19e-3826eb639b58 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.487865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5e26-c8e5-4952-b19e-3826eb639b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.513950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-cb9c5e26-c8e5-4952-b19e-3826eb639b58 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:55.515540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-cb9c5e26-c8e5-4952-b19e-3826eb639b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:55.599848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 222/457] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:55.636920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b6c88-7cb9-4515-aecf-7d87540a3eda None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:55.754377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b6c88-7cb9-4515-aecf-7d87540a3eda None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:55.766003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66d40b-8030-462d-bbc0-c32d6beed210 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.766594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66d40b-8030-462d-bbc0-c32d6beed210 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.767113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66d40b-8030-462d-bbc0-c32d6beed210 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.768916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 223/458] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:55.770159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3776ec1-ba6c-4fba-94ef-5f50d3ab0d45 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.770672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3776ec1-ba6c-4fba-94ef-5f50d3ab0d45 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.771131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3776ec1-ba6c-4fba-94ef-5f50d3ab0d45 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.772504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 224/459] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:55.777991 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.778514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.779002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.794710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:55.796129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.796548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.797141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.801637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:55.803086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:55.808421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:55.809912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-3212757c-6fe0-4203-b427-ae302b51bf66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:37:55.833921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:55.834858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:55.836309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:37:55.836309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:37:55.836309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:55.836309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:55.836309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 236/460] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:55.836817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:55.837099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:55.837368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:55.857440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:55.859140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:55.881733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 225/461] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:55.882988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:55.964701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 237/462] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:55.965995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.014916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401bd37-54ae-4a9d-897a-bf5c74a889bb None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.051493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.052557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.053070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.053542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.054473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:56.055076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.056755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.071276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 226/463] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:56.072657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.094733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5be9d-ce78-4128-a772-6b92689399d1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.128870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.130084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.130565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.131105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.142154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.144033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:37:56.161951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:56.162851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:56.163707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:37:56.163707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:37:56.163707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:56.163707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:56.166166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 238/464] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:56.167131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.190719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcdc60-d05b-41eb-a57d-b5e5f2a108a9 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.224616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.225807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.226236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.226670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.237873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.240004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:37:56.257231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:56.258078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:56.258934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:37:56.258934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:37:56.258934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:56.258934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:56.258934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 227/465] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:56.259206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.281933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774953d-ae0d-44a6-80d4-ede4f7df76ff None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.314737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9ac37-b6d1-4386-9d12-cfc508e827fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.315592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ac37-b6d1-4386-9d12-cfc508e827fb None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.315723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ac37-b6d1-4386-9d12-cfc508e827fb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.315976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ac37-b6d1-4386-9d12-cfc508e827fb None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.316770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9ac37-b6d1-4386-9d12-cfc508e827fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.318198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9ac37-b6d1-4386-9d12-cfc508e827fb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.356981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 239/466] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:56 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:56.356981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef2766-1fe9-40bb-8cbe-8fd0ad3ee65a None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.356981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef2766-1fe9-40bb-8cbe-8fd0ad3ee65a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.356981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef2766-1fe9-40bb-8cbe-8fd0ad3ee65a None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.356981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 240/467] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:56.356981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef2766-1fe9-40bb-8cbe-8fd0ad3ee65a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.369085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923263-7f1e-48c4-aee2-e6dc93e0873b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.404875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.405606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.405889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.406164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.406898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:56.407201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.408906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc947e38-ea9d-47ab-84bb-1ef558191298 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.423174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 228/468] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:56.424958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.425452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.425940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.449023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:56.450617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:56.532887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 229/469] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:56.532887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.533690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef2766-1fe9-40bb-8cbe-8fd0ad3ee65a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.586158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.586961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.587238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.587507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.588401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:56.588736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.590531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.609485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 241/470] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:56.610363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.650166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5417a17-0140-4fec-884d-6a497ea4dba0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.688098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.689066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.689489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.689923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.690765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.692356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.715898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 230/471] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:56 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:56.717274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.747694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5153b9d-6ef1-42b6-ac16-a49736faba2f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.830579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.832120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.832120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.832120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.849828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.851469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb00c4b-6fb6-4845-96bd-ef00086678d2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-097cb447-76ca-4479-a725-84d67d0163fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:37:56.870484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:56.871480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:56.872495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:37:56.872495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:37:56.872495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:56.872495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:37:56.873889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 242/472] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:56.875414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daeb3-9789-4fc8-b5be-d999e3ee0bda None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.875785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daeb3-9789-4fc8-b5be-d999e3ee0bda None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.876177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daeb3-9789-4fc8-b5be-d999e3ee0bda None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.877747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 243/473] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:56.882888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daeb3-9789-4fc8-b5be-d999e3ee0bda None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:56.915743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268a320-7bc3-4b5a-8fbb-4742103f6e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:56.916420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268a320-7bc3-4b5a-8fbb-4742103f6e9a None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.916734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268a320-7bc3-4b5a-8fbb-4742103f6e9a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.917006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268a320-7bc3-4b5a-8fbb-4742103f6e9a None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.917764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268a320-7bc3-4b5a-8fbb-4742103f6e9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:56.919248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268a320-7bc3-4b5a-8fbb-4742103f6e9a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:56.949782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 231/474] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:56 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:37:56.949782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:56.949782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:56.949782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:56.971035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:56.972324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:57.026077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daeb3-9789-4fc8-b5be-d999e3ee0bda None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:57.050880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 232/475] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:57.076295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:57.084357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:57.084357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:57.084357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:57.084357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:57.089187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:57.089796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:57.091995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:57.108903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 244/476] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:57.114250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:57.205379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b791117-2eb7-4f34-afb9-df247219b769 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:57.231939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f9a1-afe9-45b7-ac1d-ebae47b5ec10 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:57.246651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:57.247475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:57.247788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:57.248069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:57.248939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:57.251226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:57.260922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.sql.core [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 16 18:37:57.260922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 16 18:37:57.260922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [parameters: {'id': '4cecd4f115324ec48389a626d33f2e07', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 16 18:37:57.260922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91542) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 16 18:37:57.274557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:57.275489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:57.275912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:57.276422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:57.277405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:57.279055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application result = self._query(query) May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application conn.query(q) May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application result.read() May 16 18:37:57.282255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return ref.to_dict() May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 18:37:57.283569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application next(self.gen) May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application yield resource May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application next(self.gen) May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self.session.rollback() May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self.force_reraise() May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise self.value May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application session.commit() May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 16 18:37:57.284746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._prepare_impl() May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self.session.flush() May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._flush(objects) May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application compat.raise_( May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application flush_context.execute() May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rec.execute(self) May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 16 18:37:57.285812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application _emit_insert_statements( May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application c = connection._execute_20( May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ret = self._execute_context( May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 18:37:57.286875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application result = self._query(query) May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application conn.query(q) May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application result.read() May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 18:37:57.288583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [parameters: {'id': '4cecd4f115324ec48389a626d33f2e07', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:57.290124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 16 18:37:57.291634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:57.295149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 233/477] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 16 18:37:57 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:57.295965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:57.314245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 245/478] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:37:57 2022] POST /identity/v3/users => generated 249 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:37:57.434252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf0578-72f3-415a-8910-689aed9629a2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:57.479740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080b3c6-21a4-418f-9bcd-2068631ee318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:57.480998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080b3c6-21a4-418f-9bcd-2068631ee318 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:57.481512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080b3c6-21a4-418f-9bcd-2068631ee318 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:57.481982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080b3c6-21a4-418f-9bcd-2068631ee318 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:57.501581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080b3c6-21a4-418f-9bcd-2068631ee318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a080b3c6-21a4-418f-9bcd-2068631ee318 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:57.513467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:57.514404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 16 18:37:57.515206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:57.518015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 234/479] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:37:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:37:57.523820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:57.627209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990434f5-d986-4dd6-9082-b6c245ec4f19 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:57.662895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:57.663900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:57.664379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:57.664951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:57.665993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:57.666547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:57.668173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b8a9d-eaff-40e0-8af5-f2cb248599fb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:57.677317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 246/480] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:37:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:59.091975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24f47-dcf5-41cb-8622-1ca2c895fca8 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.092511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24f47-dcf5-41cb-8622-1ca2c895fca8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.092511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24f47-dcf5-41cb-8622-1ca2c895fca8 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.094153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 235/481] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:59.099638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.100047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.100209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.117313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:59.118496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:59.200785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 247/482] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:37:59.220103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d24f47-dcf5-41cb-8622-1ca2c895fca8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:59.322120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d24f47-dcf5-41cb-8622-1ca2c895fca8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:59.356303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:59.357158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.357437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.357710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.368163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:37:59.376262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:59.377182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:37:59.377983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:59.382249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 236/483] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:59.383643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:59.501622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e15fc-deb8-4274-8d57-d6c8526e6fe8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:59.536239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:59.536932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.537211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.537484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.538575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:59.538575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:59.540056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:59.554361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 248/484] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:59.555712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:59.672249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38e14-6473-46c4-87d3-339e1a23a254 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:59.709641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:59.710427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.710708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.710983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.726633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:59.728092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-fa30a789-3e69-45d1-bbf7-c337a5937940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:37:59.752565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:37:59.754575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:37:59.756550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:37:59.756550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:37:59.756550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:37:59.756550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:37:59.756550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 237/485] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:37:59.762749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:59.819775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bb6cd-5281-4eb3-9a7c-10333ff8bf82 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.820094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bb6cd-5281-4eb3-9a7c-10333ff8bf82 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.820313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bb6cd-5281-4eb3-9a7c-10333ff8bf82 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.821689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 238/486] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:37:59.826904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392a2ef-ee62-4b2d-beeb-bf62d0227b28 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.827188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392a2ef-ee62-4b2d-beeb-bf62d0227b28 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.827461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392a2ef-ee62-4b2d-beeb-bf62d0227b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.848656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d392a2ef-ee62-4b2d-beeb-bf62d0227b28 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:59.849840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d392a2ef-ee62-4b2d-beeb-bf62d0227b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:37:59.889350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b58b5-0b46-4f11-886c-74eae3713a2c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:37:59.923415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:37:59.924167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.924535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.924889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.925679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:37:59.926050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:37:59.927467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:37:59.932507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 239/487] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:37:59.934030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54246b1-be95-4954-8b75-c02a00d59345 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.934317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54246b1-be95-4954-8b75-c02a00d59345 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.934591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54246b1-be95-4954-8b75-c02a00d59345 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.937137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 240/488] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:37:59.942127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3457777-45b1-4d84-a839-655422780c7f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:37:59.942411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3457777-45b1-4d84-a839-655422780c7f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:37:59.942682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3457777-45b1-4d84-a839-655422780c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:37:59.944128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 249/489] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:37:59.949757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:37:59.959788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d3457777-45b1-4d84-a839-655422780c7f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:37:59.962121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d3457777-45b1-4d84-a839-655422780c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:00.050240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 241/490] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 16 18:38:00.051183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3457777-45b1-4d84-a839-655422780c7f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.072844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828de59-c79f-43f3-85f6-a2a01f087ed6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.108165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.109893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.110370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.110862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.118435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.120119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:00.135824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:00.137009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:00.137907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:00.145185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 250/491] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:37:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:00.146274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.165672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3457777-45b1-4d84-a839-655422780c7f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.199791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.200652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.201500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.202111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.212247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:00.220544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:00.221447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:00.222238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 242/492] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:00.222887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.251259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1376e-1c5a-45e9-8602-ba2a08cf9cd5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.289878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.290925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.291349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.291762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.305106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:00.313577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:00.314461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:00.315260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:00.318012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 251/493] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:00.319071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.327578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c55b5-e456-47f7-beb2-882684aa71c4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.361650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.362426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.362780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.363293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.364233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:00.365013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.366260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.411175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 243/494] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:00.411175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.438389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbb4c-a921-43f8-a1b6-241aefd0b9a7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.476555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.477236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.477545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.477791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.478528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:00.478831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.480196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.487945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 252/495] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:00.488836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.524382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa54df8-e570-4f54-ab2e-561fa9002a2f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.558555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.559142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.559423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.559696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.560984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:00.561287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.562655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9bbd2-4a74-4a6c-9fa4-41c75d343df2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.570427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 244/496] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:00.571709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41944756-f1d4-4efb-82d4-0ffd216dd1a7 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.571994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41944756-f1d4-4efb-82d4-0ffd216dd1a7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.572269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41944756-f1d4-4efb-82d4-0ffd216dd1a7 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.573674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 245/497] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:00.576584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41944756-f1d4-4efb-82d4-0ffd216dd1a7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.588389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ecd3f-2891-4c97-99c4-beadaa3e969d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.619419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b6711-1905-4ade-8a5c-ccd02ccefc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.620052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b6711-1905-4ade-8a5c-ccd02ccefc0e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.620328 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b6711-1905-4ade-8a5c-ccd02ccefc0e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.620609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b6711-1905-4ade-8a5c-ccd02ccefc0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.669949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41944756-f1d4-4efb-82d4-0ffd216dd1a7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.683415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b6711-1905-4ade-8a5c-ccd02ccefc0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.684904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b6711-1905-4ade-8a5c-ccd02ccefc0e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.700570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.702060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.702341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.702608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.712455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.713886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a969d206-6c9c-4dbe-ad84-debfb97e5e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:00.729774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:00.730763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:00.731852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:00.731852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:00.731852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:00.731852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:00.731852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 246/498] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:00.732292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83485f7f-4f91-440d-889d-0e48e2e9e993 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.733978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83485f7f-4f91-440d-889d-0e48e2e9e993 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.733978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83485f7f-4f91-440d-889d-0e48e2e9e993 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.733978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 247/499] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:00.736459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83485f7f-4f91-440d-889d-0e48e2e9e993 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.780423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 253/500] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:00.781739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.782025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.782302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.797759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:00.798939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:00.832315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83485f7f-4f91-440d-889d-0e48e2e9e993 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.860669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 254/501] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:00.861475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.862825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.863640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.863923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.864194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.874769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.876214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a6083f20-12e7-4733-a6f5-9dac737d6e18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:00.893468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:00.894487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:00.895472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:00.895472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:00.895472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:00.895472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:00.895472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 248/502] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:00.897072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.897500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.897779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.914028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:00.915191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:00.955268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b406f96-73bd-42d2-9698-ae6a4810cd13 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:00.979250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 249/503] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:00.980091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:00.987390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:00.988028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:00.988303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:00.988570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:00.989336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:00.989634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:00.990981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.002414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 255/504] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:01.005545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.087357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574120b3-9af3-4d75-bc2a-5ee27d47d1f6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.096617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64973e0-06cb-464b-999f-2597522bb641 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.118728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.119495 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.119772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.120042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.130336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.130994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.131499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.131982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.132401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.132641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.142302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:01.149868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.150692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.151514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:01.151514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:01.151514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:01.151514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:01.151726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.153224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:01.154045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:01.154631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 250/505] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.154704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 256/506] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.154769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.154847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.245954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7dd9a-46d9-4f9c-b6b7-563796bbc82e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.247319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e843edb-c1d2-4569-82ed-0eac3585b048 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.277170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.277957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.278236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.278444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.278551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.279594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.279868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.280131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.280854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:01.281153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.282501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.289871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.291306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.293314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 251/507] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:01.294081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:01.308404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.309381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.310252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:01.310252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:01.310252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:01.310252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:01.310252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 257/508] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.310800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.385785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1af6ac-cb85-457a-9cc7-23072d836d19 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.402598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4a2b0-9cab-43e2-a07e-939be7ac336a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.416761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.417535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.417809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.418076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.425649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.427060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.434970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.435636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.435918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.436191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.436928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:01.437235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.438584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:01.441605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.442563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:01.443429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 252/509] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.443973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.449883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 258/510] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:01.450667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.539578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a177fe-2f57-44ff-99f0-8407dae9afdb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.542274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b01622-9d89-4c70-9e34-cea0f7aabd57 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.570492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.571134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.571409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.571681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.572408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:01.572737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.574087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.574239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.575013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.575292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.575561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.581627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 253/511] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:01.582384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.582514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.583922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:01.598909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.599856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:01.600763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 259/512] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.601641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.678333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ea942-bcbf-484d-b6e2-bbd194dcf6ce None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.692284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb7b22-9dbe-424a-a710-57cafedcd3c7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.708932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.709572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.709848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.710118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.710821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:01.711121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.712467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.724218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 254/513] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:01.724334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.724840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.725060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.725243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.725509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.726329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:01.726632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.728047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.762272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 260/514] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:01.763068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.822880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93ca57-89e8-4114-939a-0d2843e7bdfb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.854538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.854968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00daa9-8d43-4758-b351-c029b28fb90a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:01.855311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.855587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.855855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.862356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.863967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:01.877748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.878568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:01.879407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 255/515] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.879862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.892134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:01.892941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:01.893221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:01.893487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:01.902898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:01.905235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:01.922162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:01.923049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:01.923873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:01.923873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:01.923873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:01.923873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:01.923873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 261/516] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:01.924602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:01.979383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c8110-adfb-4de3-83e8-8ccc27687751 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.010579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.011234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.011515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.011788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.012610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:02.012945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.014988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.017330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c6b12-5005-4ce1-a609-50931a67c244 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.031197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 256/517] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 16 18:38:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:02.031983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.052011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.052769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.053047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.053317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.054009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.055424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.074453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 262/518] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:38:01 2022] POST /identity/v3/users => generated 247 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:02.075074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.145042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7349f-5bc4-4736-90cd-3966cadeda6d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.179384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f0e32-e790-4347-9787-a02fac3e0540 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.180082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.180701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.180988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.181263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.214378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.214954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.215232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.215501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.216292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:02.216651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.218116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.236723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 263/519] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:02.237942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.250738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.252943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.344619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464459d-44ad-42c4-9aab-1ea71ce27a47 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.351426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 257/520] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:02.352347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.378075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.378748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.379055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.379325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.380056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:02.380357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.381763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.398171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 264/521] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:02.398171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.453364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d33b8-f4ba-4665-803f-0e07a48d9b60 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.489450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ca10-b946-4d2d-93ac-e5e378a03db4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.494349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.495042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.495326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.495596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.522604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.523248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.523529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.523800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.524617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=b163f5034b7a4d96a224c07b9c1428c3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:02.524955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.526397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.531632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 265/522] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/role_assignments?user.id=b163f5034b7a4d96a224c07b9c1428c3&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:02.532396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.580239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.581767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.667821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93320b2d-93c2-4a80-bffa-e6586d076d79 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.691167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 258/523] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:02.692400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.714891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.716181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.716605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.717001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.731873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.734044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:02.759122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:02.760152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:02.761213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:02.761213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:02.761213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:02.761213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:02.761429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 266/524] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:02.763644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.818408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2657710-4fba-4e5d-ab29-e49a70eaa87b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.861591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.862393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.862955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.863237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.871719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.873636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.883209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01beeaf-0534-4628-aefc-283340b3b32f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:02.891525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:02.892718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:02.894190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 259/525] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:02.895354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:02.932070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:02.933046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:02.933631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:02.934034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:02.946679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:02.948858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:02.971271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:02.972279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:02.973186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:02.973186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:02.973186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:02.973186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:02.973186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 267/526] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:02.976662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.012432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9162cca-c221-4306-9199-38e8a964810e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.061166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.061821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.062098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.062367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.063087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:03.063390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.064772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.084680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 260/527] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:03.085946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.106699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a601b-f27d-49c4-8351-792831764436 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.143487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.144361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.144664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.144971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.145815 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:03.146123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.147599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.169830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 268/528] 10.222.0.36 () {56 vars in 1139 bytes} [Mon May 16 18:38:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:03.170669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.194399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aca0d0-224e-4f69-9938-16dbf74ac91b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.228661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.229341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.229621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.229898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.230608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:03.230910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.232336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.245117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 261/529] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:03.246125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.271029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485e04-4350-4049-a419-342927a8cbe9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.305542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.306334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.306612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.306886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.313670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.315131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:03.329757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:03.331707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:03.333709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:03.340898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 269/530] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:03.340898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.347360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82b122-10a4-4551-8781-efaa7d8cf821 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.381349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.381996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.382274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.382547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.437484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e8b07-b424-496d-8809-607462f3ced3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.451814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.453317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.471893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.472699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.472982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.473253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.479855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.481325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:03.495262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:03.497147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 270/531] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:03.499694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.566577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 262/532] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:03.567645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.611641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b219d6-7111-434a-abb1-ecb4efe8e99e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.652548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.653531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.654715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.655211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.656758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:03.657299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.659726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.675966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 271/533] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:03.676994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.680236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a460640-a114-46c5-8813-ff3724cb71f1 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.720954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.722027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.723203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.723760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.724746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:03.725191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.726757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.752324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 263/534] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:03.753475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.797266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0644a7-bd27-494b-95ff-e8ae2f3dd12f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.831943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.832977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.833395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.833832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.844675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.846354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.857314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c484cfe-c4fb-48d6-9bed-04da044722cb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:03.863862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:03.864859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:03.865676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:03.865676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:03.865676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:03.865676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:03.867843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 272/535] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:03.868821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.893919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:03.894683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:03.895028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:03.895370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:03.896266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=dfc10bc677cc4cf4b673fdcbd937d352) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:03.896960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:03.898177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:03.916027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 264/536] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/role_assignments?user.id=dfc10bc677cc4cf4b673fdcbd937d352&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:03.916027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:03.986219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5defa-0f64-4a3f-ae86-a934f3918939 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:04.021628 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:04.022564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.023055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.023484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.024509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=fbdba20bd1c243248348d5c5672fefbb) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:04.025041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:04.026651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:04.033600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 273/537] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/role_assignments?user.id=fbdba20bd1c243248348d5c5672fefbb&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:04.088686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7778a5-19c6-4f6d-a3dd-8645b1f00179 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:04.142417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:04.143095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.143375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.143653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.144386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:04.144728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:04.146130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf2257-0bce-4390-a566-2efac8957960 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:04.160319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 265/538] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:04.165896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:04.308876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40d3ea-3f08-4ce5-a2bc-176f967005c5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:04.345173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:04.346382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.346851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.347312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.354552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:04.356343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:04.371849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:04.372788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:04.373611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 274/539] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:04.374478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb908cb-93a0-43a3-adc6-8d95b92691a6 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.374896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb908cb-93a0-43a3-adc6-8d95b92691a6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.374896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb908cb-93a0-43a3-adc6-8d95b92691a6 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.376184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 266/540] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:04.381005 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:04.381808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.382095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.382372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.415105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:04.416295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:04.516986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38e40-54d9-45f0-84b5-f0e2885f065b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:04.517303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 267/541] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:04.572136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369349ac-912f-4620-8791-735e994431b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:04.573180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369349ac-912f-4620-8791-735e994431b3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.573673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369349ac-912f-4620-8791-735e994431b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.574133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369349ac-912f-4620-8791-735e994431b3 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.575092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369349ac-912f-4620-8791-735e994431b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:04.575584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369349ac-912f-4620-8791-735e994431b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:04.577281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369349ac-912f-4620-8791-735e994431b3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:04.593692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 275/542] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:04.595557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:04.599934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369349ac-912f-4620-8791-735e994431b3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:04.728681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369349ac-912f-4620-8791-735e994431b3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:04.762759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:04.765865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.766403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.767640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.769480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=e653231605f345b0846ac42508e7d9a6) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:04.770027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:04.771810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:04.784162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 276/543] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:04 2022] GET /identity/v3/role_assignments?user.id=e653231605f345b0846ac42508e7d9a6&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:04.793279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c5130-4488-4b39-9b78-b553394332e8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:04.836354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececb983-6dd9-4ea0-84c7-f6f05d90875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:04.837203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececb983-6dd9-4ea0-84c7-f6f05d90875d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.837486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececb983-6dd9-4ea0-84c7-f6f05d90875d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.837762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececb983-6dd9-4ea0-84c7-f6f05d90875d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.854558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececb983-6dd9-4ea0-84c7-f6f05d90875d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-ececb983-6dd9-4ea0-84c7-f6f05d90875d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:04.862457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:04.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:04.864246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 268/544] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:04.871377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:04.890338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe83cf-30a0-4ec4-890d-d9e3b4f5a1d0 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.890658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe83cf-30a0-4ec4-890d-d9e3b4f5a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.890885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe83cf-30a0-4ec4-890d-d9e3b4f5a1d0 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.892258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 269/545] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:04.897664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:04.897945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:04.898203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:04.935330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:04.936574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:05.038459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef702d-e21d-4643-aadf-6882befa6439 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.044525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 270/546] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 16 18:38:05.083603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.094970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.095701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.095987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.096262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.097051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:05.097361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:05.098789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:05.107619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 277/547] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:05.117684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.220127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa32e3-b537-400d-9ba9-2cda3975b48f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.279819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.280616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.280916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.281191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.298041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d53dec-c973-46af-8f34-1729aaed449a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.300732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:05.313065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:05.313952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:05.314721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:05.317456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 271/548] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:05.325151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.362731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.363455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.363735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.364014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.379939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:05.381423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:05.414573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:05.415455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:05.416283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:05.416283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:05.416283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:05.416283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:05.416283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 278/549] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:05.423898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.514333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b8ff-7c74-4890-8e13-cfc3378d27c2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.573369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.574037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.574316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.574588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.575333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:05.575672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:05.577087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:05.590256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 272/550] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:05.596956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.620324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459121cd-8f97-451b-bf29-6b13347258f9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.677877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.678538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.678827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.679085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.679864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:05.680164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:05.681632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:05.694503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 279/551] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:05.705572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.741799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c855-a8e6-44ad-8401-52bedf0083d0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.789865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.790662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.790941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.791215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.814751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:05.816230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:05.836076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1670-260c-433a-885b-2d9d60323fb0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:05.845489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:05.846313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:05.847135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:05.847135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:05.847135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:05.847135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:05.847135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 273/552] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:05.853593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.885959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:05.886687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:05.886969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:05.887239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:05.899586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:05.901087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:05.927867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:05.928800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:05.929668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:05.933552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 280/553] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:05.938877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:05.956992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e56214-900f-4ca6-a304-cf436c42538a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.002257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.002927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.003209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.003482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.004203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:06.004504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.005909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.026756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 274/554] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:06.040409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.067238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4849e66-55b2-4fdf-a745-29131a4d4543 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.152123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c65ab-89fb-4341-b872-120ad5a1da60 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.153670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.155140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.155632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.156092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.157205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:06.157971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.159641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.190655 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 281/555] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 16 18:38:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:06.196328 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.203290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.204378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.205380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.207680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.221561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.223287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:06.250221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:06.251213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:06.252089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:06.255653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 275/556] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:06.264472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.332364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2695302-aaef-47e4-84ca-4146168411a9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.366604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.367270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.367530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.367790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.436150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.437550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.451351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4b7a-e558-4ad7-b51b-7637908b984e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.486290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.487309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.487783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.488270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.489376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:06.489880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.491539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.511324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 276/557] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:06.530066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.551088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 282/558] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 18:38:06.557888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.640021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63dc49-cc8f-4f92-ba31-f951a40e1804 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.666015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e36f49-26ad-4803-a53a-12068c73f1a2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.704221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.706789 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.707072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.707350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.716397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.717210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.717687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.717792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.728177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.729704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.729979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.731421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-9a7951b8-1d5b-4c87-9fd8-e8d889130445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:06.748275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:06.749150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:06.749975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:06.749975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:06.749975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:06.749975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:06.749975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 283/559] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:06.752723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3da311-502e-442b-bf02-b1b2397530ec None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.753008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3da311-502e-442b-bf02-b1b2397530ec None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.753282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3da311-502e-442b-bf02-b1b2397530ec None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.754805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 284/560] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:06.759141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3da311-502e-442b-bf02-b1b2397530ec None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c06cda81-8714-4848-96f3-9ff787472950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:06.767895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:06.768756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:06.770082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:06.770082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:06.770082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:06.770082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:06.772748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 277/561] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:06.774072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.774366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.774641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.793487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:06.794806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:06.877079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 278/562] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:06.878144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.898973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3da311-502e-442b-bf02-b1b2397530ec None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:06.940729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:06.941856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:06.942333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:06.942796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:06.943785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:06.944299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:06.945977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:06.963854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 285/563] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:06.969187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:06.984071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6471ff-e75a-4e35-855b-f48a05b6cca3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.019686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.020372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.020649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.020934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.021718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:07.022023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.023503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.035849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 279/564] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:07.040963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.075477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acb426-4d94-4b61-b144-7524ee0449af None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.111160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.111941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.112221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.112490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.123962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:07.131809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:07.132762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:07.133789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 286/565] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:07.134456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.149225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcd14b-ed62-4d66-92d0-d8a9b80d89be None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.182919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.183698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.183979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.184253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.191539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.193018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-35a07a53-39ba-4746-95af-c9c6ee07e9d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:07.208105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:07.209099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:07.210036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:07.210514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 280/566] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:07.212518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8deca6-ff58-46e8-ac78-f5b0a46db2ec None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.212900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8deca6-ff58-46e8-ac78-f5b0a46db2ec None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.213182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8deca6-ff58-46e8-ac78-f5b0a46db2ec None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.219159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 281/567] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:07.220302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8deca6-ff58-46e8-ac78-f5b0a46db2ec None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.246266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5bdf1-2ae0-4884-a873-55f9c78795f9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.280362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.281172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.281453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.281724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.288349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.289814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:07.304035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:07.305001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:07.305861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 287/568] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:07.306347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.325083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8deca6-ff58-46e8-ac78-f5b0a46db2ec None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.356914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.358046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.358325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.358598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.359329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:07.359634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.361002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c195c0-3987-4cae-935d-56a9168065e2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.368579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 282/569] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:07.369763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.370047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.370326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.386504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:07.387683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:07.398984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9209ea3-d594-4601-9f7d-adaeb0279066 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.431747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.432384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.432687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.432966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.433745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:07.434050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.435507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d766aa0-2405-4002-b27e-c18b0f1ac484 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.446353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 288/570] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:07.447698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc60f-fc23-4e19-a55d-487765a1892f None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.447991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc60f-fc23-4e19-a55d-487765a1892f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.448266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdc60f-fc23-4e19-a55d-487765a1892f None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.453511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 289/571] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:07.453511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdc60f-fc23-4e19-a55d-487765a1892f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.456773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 283/572] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:07.457664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.548546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b435c-8469-4197-8c08-a1b088c9f69e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.550447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdc60f-fc23-4e19-a55d-487765a1892f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.579279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.580047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.580320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.580600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.581378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.582014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.582300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.582571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.583336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:07.583643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.585098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.589963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.591916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.596896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 290/573] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:07.597672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-4f5f0e02-4caa-4f45-82c0-75d8b9b4e7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:07.610149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:07.611175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:07.612048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:07.612048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:07.612048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:07.612048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:07.612048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 284/574] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:07.612981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.612981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.612981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.628272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:07.629456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:07.691471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 285/575] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:07.692282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.695497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07cb9e-454f-48c0-8a84-76cab6712807 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.728325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.729012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.729297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.729568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.730393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=16b2497bffb3487bbd025f22463d5a79) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:07.730700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.732149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.737487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 291/576] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/role_assignments?user.id=16b2497bffb3487bbd025f22463d5a79&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:07.738157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.794809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927b919-3c2d-49f1-9965-fa2e20214f98 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.831692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.832422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.832728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.833003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.844747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:07.845109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b09d57-ebd2-4371-9793-87d621b18a7c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:07.853169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:07.854160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:07.855063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:07.862268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 286/577] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:07.863462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.880543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:07.881492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:07.881772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:07.882047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/b163f5034b7a4d96a224c07b9c1428c3/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:07.898025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=b163f5034b7a4d96a224c07b9c1428c3, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:07.899484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:07.919322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 292/578] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:07 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/b163f5034b7a4d96a224c07b9c1428c3/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:07.920423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:07.984673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0871c-2954-4a53-a6db-6c76d59dd999 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.023010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.023424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.023759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.024081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.024858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:08.025255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.026729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.046149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 287/579] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:08.047891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.071251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb69460-fd54-4a6d-a978-94ad62fd0b64 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.116461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.117536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.117918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.118086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.132567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:08.140951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:08.141903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:08.143153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 293/580] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:08.144056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.161006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b91bb-5f04-4337-922b-f3f23414160d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.196107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.196840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.197163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.197440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.198177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:08.198475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.199832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.208841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 288/581] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:08.209652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.255506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e927-8dbe-474d-9831-db5dd711910e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.290244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.291382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.291893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.292368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] PATH_INFO: `/v3/users/glance` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.300447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.302191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.312218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61a73e-fb63-42ca-beea-de0f33549e68 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:08.317772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:08.318673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 16 18:38:08.319615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:08.322233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 294/582] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:08.323332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.346241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.346905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.347186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.347462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.348203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:08.348506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.349929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.362877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 289/583] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:08.363787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.431454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483a6cb-1452-42b4-acb7-22daf30a3f2a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.463603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2712b5-ce73-4cca-9f4f-5f0664187e18 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.466816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.467737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.468017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.468289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.479241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.480737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.497280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.498489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.498768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:08.498940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:08.499762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:08.500656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:08.500656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:08.500656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:08.500656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:08.500877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.500877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:08.501000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 295/584] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:08.501000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.504735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.506226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.526658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 290/585] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:08.527596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.606556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e7bb3-5874-4e26-aa39-0467c702cfce None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.628169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54a94-ab8f-477b-afdf-1605aa3e3760 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.640985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.641801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.642084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.642360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.653066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.654559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.662082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.662738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.663017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.663303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.663993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:08.664294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.665678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.682029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 291/586] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:08.682874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:08.686272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:08.687110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:08.687960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:08.687960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:08.687960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:08.687960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:08.690619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 296/587] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:08.691526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.787342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c21a1-afb0-4e99-b9bd-424d864c8566 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.810062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ce547-685c-46c8-a33f-7c0d571fefe1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.824326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.825575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.826017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.826291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.841912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.841912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.856039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:08.857467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:08.857754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:08.858026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:08.859926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:08.860993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:08.862085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:08.862085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:08.862085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:08.862085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:08.863618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 292/588] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:08.864705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.865765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:08.867210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:08.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:08.882813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:08.883789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 297/589] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:08.884572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:08.970122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397062b8-0e1e-4168-9050-86bc1acb547e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:08.990719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7b4e-5c7a-4197-857d-34620cd5bcdb None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.005468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.006241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.006623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.006998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.007805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:09.008208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.009704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.021513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 293/590] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:09.023843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.024723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.025010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.025282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.026322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:09.026630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.026853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.027999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.041568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 298/591] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:09.042882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.131444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420d4ac-cddf-4ef7-ba46-894a7597b0c7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.144301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ae8c-9b5b-4756-9b78-5e45e5cdc163 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.168303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.169162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.169567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.169941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.170898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:09.171297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.172847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.178523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.179327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.179608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.179882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.187875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.189367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.193844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 294/592] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:09.199020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:09.205438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:09.207429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 299/593] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:09.209739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.305398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c311d2-354d-41b9-8ef1-6797e4bcd3a4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.306234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19def5fb-ebd7-44ab-8c18-8946a7178e3a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.338871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.339524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.339696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.339874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.340145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.340575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.340984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.341906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.348578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.350101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.350723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.352116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:09.363993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:09.365768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 295/594] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:09.368030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:09.369364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:09.371000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:09.376886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:09.376886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:09.376886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:09.376886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:09.376886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 300/595] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:09.382314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.473313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1731700-65fa-4215-84d6-16566ea1cf60 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.506875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.507780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.507980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab67c0-a1bd-4cc5-8d7f-0a80192da8e7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.508727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.509147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.510147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:09.510601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.512162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.530432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 296/596] 10.222.0.36 () {56 vars in 1139 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:09.531475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.555723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.556471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.556793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.557071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.558098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:09.558413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.560043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.584785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 301/597] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:09.585981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.661777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289d03-c5bd-4676-bb8a-280f303d850c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.686749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e4d7-4b6e-4238-91bc-ee5e0a5116f0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.704549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.705246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.705575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.705856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.706637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:09.706945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.708851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.722772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.728334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.728493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 297/598] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:09.728493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.728699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.728967 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.743229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.744970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:09.767350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:09.768543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:09.769723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:09.769723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:09.769723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:09.769723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:09.772708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 302/599] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:09.773676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.850039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc42d73-ea5d-4836-b3db-88732f5164ce None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.883323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e9e9d-2c68-4d95-ac3a-47a4f5f1c615 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:09.891672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.893143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.893596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.894024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.903331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.905955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.919539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:09.920588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:09.920989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:09.921358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/fbdba20bd1c243248348d5c5672fefbb/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:09.926168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:09.927235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:09.928324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:09.930458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 298/600] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:09.931399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:09.943998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=fbdba20bd1c243248348d5c5672fefbb, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:09.945802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:09.968661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 303/601] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:09 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/fbdba20bd1c243248348d5c5672fefbb/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:09.968661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.087519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394b83-38ce-46d2-9d1b-5826e430653f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.128544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.129308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.129690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.130367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.131216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:10.131787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.133317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:10.147960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 299/602] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:10.153643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.198463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3515d1e-2fe3-4a02-8ff7-0c9ea9b7a822 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.244067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1676c-a064-499c-91b3-995514832705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.244994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1676c-a064-499c-91b3-995514832705 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.245373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1676c-a064-499c-91b3-995514832705 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.246056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1676c-a064-499c-91b3-995514832705 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.247246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1676c-a064-499c-91b3-995514832705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:10.248191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1676c-a064-499c-91b3-995514832705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.250354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1676c-a064-499c-91b3-995514832705 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:10.266462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 304/603] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:10.268156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e7244-172b-4195-a5d6-3e261b65c729 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.269358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859224-0f66-4375-b36d-1dbabbd16d75 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.272954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e7244-172b-4195-a5d6-3e261b65c729 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.274072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e7244-172b-4195-a5d6-3e261b65c729 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.275824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 305/604] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:10.276756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e7244-172b-4195-a5d6-3e261b65c729 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.306007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.306788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.307062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.307335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.314515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.315942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-eeade2ec-ed16-4e67-a303-7da1924fb011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:10.330332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:10.331239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:10.332124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 300/605] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:10.333086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9308c-e2a7-46b3-9944-e1832343ecb7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.333269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9308c-e2a7-46b3-9944-e1832343ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.333542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9308c-e2a7-46b3-9944-e1832343ecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.349060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-bad9308c-e2a7-46b3-9944-e1832343ecb7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:10.350215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-bad9308c-e2a7-46b3-9944-e1832343ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:10.383414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e7244-172b-4195-a5d6-3e261b65c729 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.424662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 301/606] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:10.424662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87291cbb-9b95-49b9-bf79-ce18f74c6e0a None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.424662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87291cbb-9b95-49b9-bf79-ce18f74c6e0a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.425051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa573dd-811b-4149-95e5-9dc7be16b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.425154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87291cbb-9b95-49b9-bf79-ce18f74c6e0a None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.425236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa573dd-811b-4149-95e5-9dc7be16b722 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.425236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa573dd-811b-4149-95e5-9dc7be16b722 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.425236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa573dd-811b-4149-95e5-9dc7be16b722 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/dfc10bc677cc4cf4b673fdcbd937d352/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.425461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 302/607] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:10.425461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87291cbb-9b95-49b9-bf79-ce18f74c6e0a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.442065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa573dd-811b-4149-95e5-9dc7be16b722 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=dfc10bc677cc4cf4b673fdcbd937d352, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.443507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa573dd-811b-4149-95e5-9dc7be16b722 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:10.459146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 306/608] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:10 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/dfc10bc677cc4cf4b673fdcbd937d352/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:10.460487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.460876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.461246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.485005 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:10.486344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:10.556793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87291cbb-9b95-49b9-bf79-ce18f74c6e0a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.565008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 307/609] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:10.566633 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.596997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.597859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.598146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.598420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.599250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:10.600726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.601686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:10.618329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 303/610] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:10.621226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.710315 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb202599-8d54-4464-8960-55081da5a0ad None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.738255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c7d1f-8716-4852-9348-51eecba11867 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.755654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.756927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.757429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.757921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.771274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:10.772193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.773026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.773305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.773631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:10.781247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:10.782206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:10.783312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:10.785900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 308/611] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:10.791289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.792592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:10.801181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:10.802174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:10.803014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 304/612] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:10.808517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.911890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8ece-d50f-42b5-8cf2-bf81baea7089 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.922289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee94a9-0fb8-479e-8eab-706d6100be16 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:10.945700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.946462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.946811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.947180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.947992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:10.948367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.949899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:10.963666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:10.968698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 305/613] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:10.968698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:10.969061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:10.969636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:10.971730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/e653231605f345b0846ac42508e7d9a6/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:10.994122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=e653231605f345b0846ac42508e7d9a6, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:10.996487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.018414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 309/614] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:10 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/e653231605f345b0846ac42508e7d9a6/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:11.019373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.145766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffddc4-be9d-4c7b-807f-0357e8148d0a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.209302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cc73c-286b-4746-91e8-f6a8dfbbf23d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.248574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.249216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.249917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.249917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.250541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:11.251232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.251853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.252178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.252689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.252960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.253231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.265357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.267106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.268085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 306/615] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:11.273108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:11.292664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:11.293593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:11.295044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:11.295044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:11.295044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:11.295044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:11.295044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 310/616] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:11.300085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.407239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9034ca5-86a9-49d7-9c94-c9fb738636aa None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.442395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.443059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.443342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.444009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.444792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:11.445106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.446483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.459882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 311/617] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:11.465074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.506828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728832e-002a-447e-9ebe-04382e3410ba None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.574651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff42244-1009-41ab-897e-4ae83d682360 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.593313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.594116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.594398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.594668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.608338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.609150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.609424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.609692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] PATH_INFO: `/v3/users/swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.614535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.615973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.617442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.618919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:11.633306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:11.634259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:11.635183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 312/618] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:11.640774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:11.647678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:11.648535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:11.649447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:11.649447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:11.649447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:11.649447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:11.661323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 307/619] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:11.666529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.788566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd2fdd-789c-4a30-8437-f4f0147a174b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.844367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca99b3c-0d31-45fa-91a1-bd43a171a6d9 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:11.858362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.859055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.859309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.859582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.860424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:11.860761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.862213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.880075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:11.881043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:11.881471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:11.881902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:11.882766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:11.883213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:11.884757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:11.896884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 313/620] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:11.898097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 308/621] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:11.902020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:11.903171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.067916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da67de-94f2-4582-b541-e34c2d352678 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.084243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f02ca-daa4-4a00-9e29-ec09f6279576 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.111383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.112166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.112441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.112738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.125860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.127290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:12.151931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:12.152918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:12.153936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 309/622] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:12.160467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.161461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.161833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.162339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.162813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.186698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.188513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:12.224217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:12.225202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:12.226083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:12.226083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:12.226083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:12.226083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:12.232419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 314/623] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:12.239058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.345236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b5999-eb58-4486-b307-d87b793ff77c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.386466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.387526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.388051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.388540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.389694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:12.390447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.392742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.426900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 310/624] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:12.430787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.456855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bf26-c050-43a9-9768-baefd39d5e62 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.523618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.525185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.525970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.526454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.527853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:12.529184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.531251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88631729-aa4d-481e-8ed5-141436b89437 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.546411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 315/625] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:12.548753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ed49-ccfe-46e3-85a6-da8b6b95bf7e None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.549604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ed49-ccfe-46e3-85a6-da8b6b95bf7e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.550407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ed49-ccfe-46e3-85a6-da8b6b95bf7e None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.553495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 316/626] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:12.554600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ed49-ccfe-46e3-85a6-da8b6b95bf7e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.590686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248db71-fa69-4809-a5c8-c4c4b975b14a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.629720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.630728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.631221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.631702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.677623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ed49-ccfe-46e3-85a6-da8b6b95bf7e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.705561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.708509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.718509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.719733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.720036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.720313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.730157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.732300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-354e6eb3-a7b0-4d3a-b1ff-d842f5c0ecfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:12.753396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:12.754369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:12.755248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 317/627] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:12.756405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.756713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.756987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.778968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:12.780431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:12.835011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 311/628] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:12.836269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.850368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 318/629] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:12.851383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:12.944864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd318a-8d27-4ff9-8fff-a1853ab2d6fb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.957932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84015bb0-5c98-469e-a920-2ee8633d70a6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:12.983833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.984503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.984819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.985097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.985876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:12.986182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:12.987666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:12.992852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:12.993642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:12.993919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:12.994194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:12.999722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 312/630] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:13.000750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.005666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.007311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-a9bbc0e9-edf9-4353-a8b8-119cd52114af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:13.025280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:13.026504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:13.027412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:13.027412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:13.027412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:13.027412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:13.027412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 319/631] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:13.028063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f94c4-8896-4dad-930a-1a03a0dfa425 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.028344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f94c4-8896-4dad-930a-1a03a0dfa425 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.028672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f94c4-8896-4dad-930a-1a03a0dfa425 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.032890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 320/632] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:13.032890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f94c4-8896-4dad-930a-1a03a0dfa425 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.109372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da2aa8-1eb7-4173-bdaa-f00a7f8ab1a0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.129811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f94c4-8896-4dad-930a-1a03a0dfa425 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.142017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.142795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.143074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.143344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.154170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:13.162148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:13.163058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:13.163993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 313/633] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:13.164715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.168100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3d50c-5a50-4bad-ab45-c83e8c5a1e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.169014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3d50c-5a50-4bad-ab45-c83e8c5a1e9d None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.169782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3d50c-5a50-4bad-ab45-c83e8c5a1e9d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.170055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3d50c-5a50-4bad-ab45-c83e8c5a1e9d None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/16b2497bffb3487bbd025f22463d5a79/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.185715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3d50c-5a50-4bad-ab45-c83e8c5a1e9d None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=16b2497bffb3487bbd025f22463d5a79, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.187163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3d50c-5a50-4bad-ab45-c83e8c5a1e9d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.202454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 321/634] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:13 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/16b2497bffb3487bbd025f22463d5a79/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:13.203602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.203890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.204166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.221478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:13.222854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:13.310680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba474d-bb80-40d1-99b5-74c0193d5102 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.317597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 322/635] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:13.318826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.373523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.374251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.374477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.374753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.375481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:13.375786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.377212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.391015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 314/636] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:13.396113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.437402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87052-854d-45eb-8a92-a7fd89bf8736 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.478847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.479434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.479709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.479977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.480748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:13.481053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.483392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.491525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 323/637] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:13.492398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.507056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926b660-0d9e-40af-b864-0b67147624a9 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.539785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464266a-5139-4ca5-a799-00349981bb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.540568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464266a-5139-4ca5-a799-00349981bb6d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.540872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464266a-5139-4ca5-a799-00349981bb6d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.541144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464266a-5139-4ca5-a799-00349981bb6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.554457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464266a-5139-4ca5-a799-00349981bb6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-5464266a-5139-4ca5-a799-00349981bb6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:13.562270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:13.563257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:13.564074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 315/638] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:13.565262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed803b25-c214-4153-bca9-3d5bf2953599 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.565262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed803b25-c214-4153-bca9-3d5bf2953599 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.565607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed803b25-c214-4153-bca9-3d5bf2953599 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.571832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 316/639] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:13.571832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed803b25-c214-4153-bca9-3d5bf2953599 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.603086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1a1fc-bf63-4ee5-883f-df22e9f04ea3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.636446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.637263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.637564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.638348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.645045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.646495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:13.661353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:13.662287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:13.663188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 324/640] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:13.664239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.674968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed803b25-c214-4153-bca9-3d5bf2953599 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.706956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.707740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.708019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.708292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.719160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.720593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-042922d7-6a5c-4fd9-b1cb-275e39e60f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:13.737196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:13.738308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:13.739283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:13.739283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:13.739283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:13.739283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:13.739283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 317/641] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:13.739889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.740173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.740446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.756499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:13.757696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:13.775438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1d35-9105-4c7d-9409-0556f9b92f16 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.808552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.809244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.809540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.809828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.810557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:13.810856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.812239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.820074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 325/642] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:13.832906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 318/643] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:13.832906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.833125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.933943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61312e3-26d7-4d07-aeef-c8817bcf5def None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.935710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d995d-b5a5-437c-81f0-f177679adda7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:13.970196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.970924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.971206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.971480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.972184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:13.972480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.973588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:13.973887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.974237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:13.974515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:13.974789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:13.975563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:13.975878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:13.977324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:13.989237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 319/644] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:13.989237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:13.994061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 326/645] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:13.994957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.091737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d254818-f289-4e86-be59-4d497c2ff4b2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.091970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b074-0c65-4ea3-b604-0ab5bbadff90 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.123720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.124015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.124510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.124722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.124831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.125005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.125111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.125268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.134959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:14.135239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.136708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:14.143233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:14.144363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:14.145453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 327/646] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:14.146181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:14.158709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:14.159611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:14.160473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:14.160473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:14.160473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:14.160473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:14.160473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 320/647] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:14.160799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.257393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed910f1d-5f04-49b7-aad7-46e2b4dbed1b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.272010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adaf4f-fedd-4eff-9d22-90cc2e852a6b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.290079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.291146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.291416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.298941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.300374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.304078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.304756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.305043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.305316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.306131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=b163f5034b7a4d96a224c07b9c1428c3) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:14.306442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.307858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.324667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 321/648] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/role_assignments?user.id=b163f5034b7a4d96a224c07b9c1428c3&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:14.324868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:14.326323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 16 18:38:14.327329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:14.327863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.327938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 328/649] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:14.327938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.430247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec80497-1653-4895-9d51-65c085a9dc7e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.436959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71617f-2b7f-4acb-8e8d-b144fe207090 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.466644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.467325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.467674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.467957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.468751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:14.469062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.470456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.474796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.475475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.475754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.476028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.477341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:14.477876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.479289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.479748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 322/650] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:14.480689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.499280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 329/651] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:14.500282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.586819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0164bb1-395e-4261-bb07-aa0b4385cf9a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.604461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bb43f-b3fc-431e-9978-69ddacbaf2bb None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.627815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.628482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.628768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.629039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.629859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:14.630168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.631590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.648979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.649780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.650060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.650334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.656267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 323/652] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:14.656267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.667167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.668624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:14.686925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:14.687858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:14.689449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:14.689449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:14.689449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:14.689449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:14.689449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 330/653] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:14.689449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.785879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e08269-8582-4f27-a51b-74d553b59d9d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.789786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60fdb4-70c4-468c-b829-172dbdaa3b80 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:14.824078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.824705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.825061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.825460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.830686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:14.831694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:14.832228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:14.832716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:14.843060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.845240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:14.865896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:14.866887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:14.867842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:14.870695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 324/654] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:14.872025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:14.895616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:14.897272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:14.995913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7035e-34c4-4b54-beb3-ff613f700c54 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.022444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 331/655] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:15.022444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.037557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.038861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.039351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.040000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.041255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:15.041771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.043371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.059990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 325/656] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:15.061246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.136455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c115c2-408b-4028-a1a3-32d0f6c2d02b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.165484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07811-55b5-4a46-8296-8329c4e51e9a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.180616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.181726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.182417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.182842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.183835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:15.184301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.186657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.199781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.201279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.201658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.202029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.213677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 332/657] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:15.213677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.217125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.219490 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:15.236983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:15.238088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:15.239197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:15.239197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:15.239197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:15.239197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:15.244803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 326/658] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:15.244803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.343475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8c158-f132-4562-b23e-e354841c74b3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.356976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10562-7188-45a1-9f88-2ce8648145f2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.388578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.389777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.390312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.390791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.391029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.391828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.392268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.392647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.400008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.402133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:15.422179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:15.423160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 16 18:38:15.424094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:15.426866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 333/659] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:15.427955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.463665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.465330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.541795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e886ac-869a-455c-9084-271ee17c4911 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.562740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 327/660] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:15.563826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.580912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.581578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.581856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.582131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.582856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:15.583162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.584533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb78ae-d547-4f54-938a-dc2a634b6ce2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.595438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 334/661] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:15.596753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578f2ab-7c2b-462b-97b9-95e916a944d5 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.597041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578f2ab-7c2b-462b-97b9-95e916a944d5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.597319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578f2ab-7c2b-462b-97b9-95e916a944d5 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.600672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 335/662] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:15.600672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578f2ab-7c2b-462b-97b9-95e916a944d5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.669701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe841e-c977-4dbc-b560-a63b7f7ae8e7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.697569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578f2ab-7c2b-462b-97b9-95e916a944d5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.704039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.704720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.705009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.705279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.706102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:15.706407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.707838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.728786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 328/663] 10.222.0.36 () {56 vars in 1139 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:15.728786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.738953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.739738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.740019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.740301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.751165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.752605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-c4330590-047e-46cc-a9fd-1722ece1a96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:15.769466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:15.770357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:15.771237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:15.771237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:15.771237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:15.771237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:15.771237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 336/664] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:15.772001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.772102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.772362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.789149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:15.790415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:15.823439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ec733-dc9c-4388-8f18-61f588547225 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.856838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:15.864799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 337/665] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:15.864924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:15.864924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:15.864924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:15.865108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.878727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:15.880170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:15.894997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:15.895869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:15.897167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:15.900928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 329/666] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:15.901621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:15.964224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885eacd-72da-4363-a8f2-854961ec3435 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:15.999572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e7664-2c17-4f9b-a8e0-ecbd9f6f9a3d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.000168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.000898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.001184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.001453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.031524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.032178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.032560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.033028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.033799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.034098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.035446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.052215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 330/667] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.052215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.075726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.077221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.143651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea98f8-c0e8-4aa5-8831-79c6e805959d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.176292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.177111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.177395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.177672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.182533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 338/668] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:16.182533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.193021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:16.200848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:16.201811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:16.202646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 331/669] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:16.203324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.279873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec61505-b939-4827-982e-d8706a5d3807 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.302595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed0474-3ded-4d22-b54d-f39a9a659f0d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.312847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.313638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.314008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.314371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.315235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.315638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.317208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.333429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 339/670] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.333429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.336365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.337312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.337593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.337865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.345190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.346623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:16.360974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:16.361837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:16.362726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:16.366977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 332/671] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:16.368334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.429358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706248-f69b-4eda-b048-be79eeef93ef None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.460930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.461712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.461987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.462258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.464265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8aa5-7ac8-490d-964f-177f2ec36bc3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.472847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.474287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:16.491635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:16.492826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:16.493765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:16.493765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:16.493765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:16.493765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:16.493765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 340/672] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:16.493765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.502223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.502904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.503180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.503449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.504200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.504501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.505910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.520224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 333/673] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.520224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.608214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57306-e796-4d52-a2ec-a389c7d60b1b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.611743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1056d-fd4f-4687-9020-116745521ddf None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.643081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.643722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.644000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.644268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.644575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.645115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=fbdba20bd1c243248348d5c5672fefbb) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.645278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.645445 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.645544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.645795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.646573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.646835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.646938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.648276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.660808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 341/674] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/role_assignments?user.id=fbdba20bd1c243248348d5c5672fefbb&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.660808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.661424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 334/675] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.662218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.768864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe609f1-2b29-441e-b18f-cac5ddba1719 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.777591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7431a-e807-404c-9b64-a9ff9955b0b0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.804942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.806030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.806314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.806588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.812929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.814394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.814675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.814948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.815683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.815986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.817439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.817554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.819018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.835829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 342/676] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.835829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-cd0cab2d-9b86-4689-a18e-dbcd418ca635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:16.837781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:16.838668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:16.839548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:16.839548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:16.839548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:16.839548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:16.844134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 335/677] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:16.845540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcce9f-7b04-4308-8f2d-ec43ae565982 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.846351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcce9f-7b04-4308-8f2d-ec43ae565982 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.846351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcce9f-7b04-4308-8f2d-ec43ae565982 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.849259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 336/678] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:16.849374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcce9f-7b04-4308-8f2d-ec43ae565982 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:16.950887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8b32f-553e-4a00-9f7c-c15fccfb04f6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.959330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcce9f-7b04-4308-8f2d-ec43ae565982 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:16.987676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.988329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.988606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.988911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.989728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=dfc10bc677cc4cf4b673fdcbd937d352) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:16.990036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:16.991454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:16.992907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:16.993688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:16.993969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:16.994246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:16.997836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 343/679] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/role_assignments?user.id=dfc10bc677cc4cf4b673fdcbd937d352&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:16.998898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.001603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.003083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-9077bace-f882-40b8-9763-f20b794bbb6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:17.019713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:17.020722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:17.021618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:17.025826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 337/680] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:17.027296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.027619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.027904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.047101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:17.048322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:17.121891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 338/681] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:17.123422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.142895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b41bed-4084-43ee-af8f-9f75106934a8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.184130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.185732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.186220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.186713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.187635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:17.188143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.189761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.202959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 344/682] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:17.204252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.236331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37574d-ac87-4c50-97ad-084b7a1825c3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.281254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.281917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.282191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.282465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.283503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:17.283812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.285319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.297608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 339/683] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:17.298469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.320178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372b1c-8bba-4cf7-a5fc-940081798198 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.363336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.364286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.365077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.365359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.378594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:17.388925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:17.389881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:17.390709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:17.391868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 345/684] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:17.391950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.402036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4ed95-0f90-4a58-9fa8-50504c96029a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.437014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.437807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.438139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.438416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] PATH_INFO: `/v3/users/swift` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.446376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.447880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:17.462554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:17.464860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 340/685] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:17.467224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.498802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c4d7a-f0fe-401e-b8ca-1afa286e9d74 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.532173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.533078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.533358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.533632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.534451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=e653231605f345b0846ac42508e7d9a6) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:17.534758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.536311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.549171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 346/686] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/role_assignments?user.id=e653231605f345b0846ac42508e7d9a6&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:17.550192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.566547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38451f-d1fc-4f3c-9c3b-7e4b5d79a52a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.603441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.604187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.604472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.604815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.605640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:17.605948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.607896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.617344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 341/687] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:17.623382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.714808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614808-b962-4d49-81ad-0f5ce8c12515 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.728398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f7eb-515a-4576-b1cd-06b3c380fe78 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.763971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.764808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.765095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.765376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.768795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.769661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.770092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.770511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.771504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:17.771964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.773736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.776525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.778019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.794881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 347/688] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:17.799353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:17.800436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:17.801533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:17.801533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:17.801533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:17.801533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:17.801533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 342/689] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:17.803171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.804443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:17.907165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbbae1-ab7d-45a0-86b4-1a674b71bf2d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.910911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38ad7d-9146-45dd-8064-7818775aeffa None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:17.943554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.944466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.944912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.945354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.945798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:17.946473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:17.946765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:17.947030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:17.947762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:17.948068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:17.949495 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:17.963173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 343/690] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:17.969174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.020936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.022395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.071536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a333d-bb20-476e-9439-8431d77dba3b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:18.105914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:18.107250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:18.107530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:18.107803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:18.115080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.116609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.125028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 348/691] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:18.130583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:18.131188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:18.133365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:18.135697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:18.141369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 344/692] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:38:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:18.142143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.246638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db394-d042-4722-85f5-6f2a9a287458 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:18.253837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f447c61-1e0f-4063-9ac6-5c4bd88532cb None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:18.284529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:18.285258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:18.285536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:18.285806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:18.286630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:18.286931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.288345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.295200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:18.296150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:18.296420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:18.296723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:18.307090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 345/693] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:18.312941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.317873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.320217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:18.352905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:18.354984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:18.357254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:18.357254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:18.357254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:18.357254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:18.357254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 349/694] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:18.365951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.501074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdb25b-a009-4a76-a71c-d75702b1b932 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:18.542285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8b843-462c-4170-a6d4-82ddab22ae6d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:18.561869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:18.562523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:18.562804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:18.563107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:18.607244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:18.608402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:18.608782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:18.609062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:18.610771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:18.611339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.613246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.639991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 350/695] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:18.645373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.688106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.689589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.768797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433bba-6362-4f96-89ef-f6bef16df046 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:18.807867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:18.808680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:18.808962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:18.809234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:18.821977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:18.823447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:18.840248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:18.841293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:18.842559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 351/696] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:18.849750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.884154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 346/697] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 16 18:38:18.890040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:18.991628 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e37fa-8db3-489c-b794-2e43086cbdef None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.035105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.035779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.036287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.036447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.037255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:19.037549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.040847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.071175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 352/698] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:19.076894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.091177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4deaf-e2f1-4c8a-b8e3-d389354480fb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.128552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.129382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.129815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.130221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.141293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.142950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-8e15c723-b8ac-4da5-953e-5a0fd7163363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:19.162173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:19.163137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:19.164683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:19.164683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:19.164683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:19.164683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:19.164683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 347/699] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:19.167890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04472a31-dd08-4b9b-bd00-bb7e5fa9d338 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.168292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04472a31-dd08-4b9b-bd00-bb7e5fa9d338 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.168692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04472a31-dd08-4b9b-bd00-bb7e5fa9d338 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.171402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 348/700] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:19.172195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04472a31-dd08-4b9b-bd00-bb7e5fa9d338 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.245086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724800b-9ed5-4868-b03a-d0c805eadbfa None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.275868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04472a31-dd08-4b9b-bd00-bb7e5fa9d338 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.285930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.286508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.286783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.287051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.288236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=16b2497bffb3487bbd025f22463d5a79) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:19.288545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.290623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa0fa3-a778-4631-b58e-608b31b4c293 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.300759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 353/701] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/role_assignments?user.id=16b2497bffb3487bbd025f22463d5a79&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:19.302052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.302342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.302617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.332957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.334050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.335640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.336194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.336809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:19.337964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:19.339078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:19.339600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.343467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.364577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 349/702] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:19.370063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.485889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 354/703] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:19.545076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43ea59-6203-4312-99b6-d5f45ce46776 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.546794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.586472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.587559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.588007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.588420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.596471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.598170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2fbdc331-0c05-4d30-b94b-cb7e415f91f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:19.620563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:19.621529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:19.622647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 350/704] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:19.623675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004f286-c333-492f-a6c0-6b41bac3803b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.624027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004f286-c333-492f-a6c0-6b41bac3803b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.624366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004f286-c333-492f-a6c0-6b41bac3803b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.625836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 351/705] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:19.631662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.631951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.632221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.648911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:19.650146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:19.672939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fdc7d-5292-4760-8339-557cd4a21409 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.718071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.718867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.719135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.719399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.730805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.732504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.732644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 352/706] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:19.733235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:19.761233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:19.762195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:19.764017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:19.764017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:19.764017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:19.764017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:19.764017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 355/707] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:19.764017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.854861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d1ec6-7f4a-4747-9449-bc7b8b524984 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.866090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d4da4-25d7-4a0e-8e53-e05618aa03b9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:19.888292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.888980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.889252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.889513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.890282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:19.890583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.892001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.901247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:19.902052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:19.902327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:19.902600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:19.903326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:19.905233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:19.910160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 353/708] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:19.911102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:19.924668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:19.926979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:19.928258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 356/709] 10.222.0.36 () {56 vars in 1129 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:19.929313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:20.015511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f63f1-c34f-4295-b0b5-05114a296e9c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:20.035308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701953f1-2ae9-4981-b875-e224e7bcf284 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:20.050899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:20.051766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.052045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.052314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.053059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:20.053364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:20.054849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:20.067313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 354/710] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:20.068077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:20.074324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:20.075256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.075548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.075915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.076871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=b163f5034b7a4d96a224c07b9c1428c3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:20.077288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:20.078827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:20.085352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 357/711] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:19 2022] GET /identity/v3/role_assignments?user.id=b163f5034b7a4d96a224c07b9c1428c3&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:20.086334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:20.265745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9340c-f9dc-4feb-8c0a-10f969930cf2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:20.311205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:20.312034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.313948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.314107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d316-7532-4a89-bc47-0d4c543d8266 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:20.314217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.314958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:20.315264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:20.316661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:20.331231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 355/712] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 16 18:38:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:20.336940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:20.361470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7179e8-a0a0-425d-82cc-1c1f9586a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:20.362309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7179e8-a0a0-425d-82cc-1c1f9586a422 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.362685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7179e8-a0a0-425d-82cc-1c1f9586a422 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.363055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7179e8-a0a0-425d-82cc-1c1f9586a422 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.363864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7179e8-a0a0-425d-82cc-1c1f9586a422 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:20.365442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7179e8-a0a0-425d-82cc-1c1f9586a422 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:20.390678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 358/713] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:38:20 2022] POST /identity/v3/users => generated 254 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:20.480737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52934-99b0-4b77-90ea-60c3d91563da None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:20.518074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:20.518722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.518999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.519313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.520033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:20.520335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:20.521779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee37c-b1a2-4a0f-97c9-b40cc62830fd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:20.529020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 356/714] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 16 18:38:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:20.560257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f17fa-30b9-4dc7-a4ed-ebca16665839 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.560566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f17fa-30b9-4dc7-a4ed-ebca16665839 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.560853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f17fa-30b9-4dc7-a4ed-ebca16665839 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.562559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 359/715] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:20.568231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.568548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.568841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.587559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:20.588806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:20.661316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 357/716] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:20.682987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f17fa-30b9-4dc7-a4ed-ebca16665839 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:20.807925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f17fa-30b9-4dc7-a4ed-ebca16665839 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:20.847601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:20.848389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:20.848694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:20.848961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:20.849606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:20.850965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:20.862952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:20.863998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:20.865073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 360/717] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:20 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:20.873860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:21.028693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d5cf-ea3f-423e-8684-4d61e803ef3d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:21.068312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:21.068930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:21.069214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:21.069492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:21.070209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] RBAC: Adding query filter params (name=network) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:21.070516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:21.071878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35395e-7bf7-4a92-91d1-95ed6fecee31 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:21.077166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 358/718] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:21.082653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:21.210799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe45d51-96f1-4b26-b31a-b673776ef0d5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:21.248705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:21.249371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:21.249653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:21.249928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:21.250637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] RBAC: Adding query filter params (type=network) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:21.250941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:21.252304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d993f37-fd62-4ec4-8179-7dbbe4d8afde None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:21.261178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 361/719] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:22.256812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e006f5-54b8-4a20-b0fb-1ed265c0c0ce None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:22.257308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e006f5-54b8-4a20-b0fb-1ed265c0c0ce None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:22.257308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e006f5-54b8-4a20-b0fb-1ed265c0c0ce None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:22.258869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 359/720] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:22.263992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:22.264432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:22.264620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:22.285542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:22.286719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:22.373513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 362/721] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:22.408939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e006f5-54b8-4a20-b0fb-1ed265c0c0ce None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:22.538141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e006f5-54b8-4a20-b0fb-1ed265c0c0ce None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:22.586248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89c008-3de0-4ab7-afe7-07a5ace39b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:22.587032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c008-3de0-4ab7-afe7-07a5ace39b0c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:22.587305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c008-3de0-4ab7-afe7-07a5ace39b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:22.587571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c008-3de0-4ab7-afe7-07a5ace39b0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:22.605168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89c008-3de0-4ab7-afe7-07a5ace39b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-fe89c008-3de0-4ab7-afe7-07a5ace39b0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:22.613150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:22.614141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:22.615544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 360/722] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:22.620057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:22.742698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b3f08-8fca-4b43-8687-ad3e191afe1b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:22.743020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b3f08-8fca-4b43-8687-ad3e191afe1b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:22.743249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b3f08-8fca-4b43-8687-ad3e191afe1b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:22.744585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 361/723] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:22.747984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b83ac-ce3e-4ae0-a5c4-0b57e391b268 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:22.750606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:22.750881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:22.751150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:22.777361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:22.779109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:22.826516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:22.827203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:22.827575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:22.827920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:22.828768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:22.829139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:22.830605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:22.847798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 363/724] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:22.853307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:22.861121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 362/725] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 16 18:38:22.886313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:22.995445 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e42f-934c-4118-93dc-72bc39d429ee None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.029226 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.030013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.030287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.030553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.041575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.043018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.047488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccc49f-0c62-4663-9318-3c2e562a64a1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:23.061150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:23.062101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:23.063470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:23.063470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:23.063470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:23.063470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:23.063470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 364/726] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:23.068660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.116867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.117575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.117856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.118122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.143866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:23.157420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:23.159917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:23.161970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:23.168416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 363/727] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:23.178696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.242196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb002df3-3edf-4f98-9551-9f8908c35d0b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.277106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.277891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.278108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.280679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.280679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:23.280679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.281173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.293158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 365/728] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:23.298263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.370084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f08f0-ad4d-42ec-accd-02407745f92c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.430381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.431046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.431323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.431593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.432410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:23.432623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.434096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d6e9f-73e6-48ee-b137-14e8a2b9b084 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.442980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 364/729] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:23.448851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14633d-6fcc-4bae-8d5a-5e9acfbb0a86 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.449092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14633d-6fcc-4bae-8d5a-5e9acfbb0a86 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.449349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14633d-6fcc-4bae-8d5a-5e9acfbb0a86 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.454678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 365/730] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:23.454678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14633d-6fcc-4bae-8d5a-5e9acfbb0a86 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.476027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a515dc1-62e9-4535-b4b4-82294e4b9d82 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.511296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.512083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.512358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.512814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] PATH_INFO: `/v3/users/swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.519808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.521328 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-020d67d7-26a4-4453-9374-96b2aabef01b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:23.535758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:23.536733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:23.537687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 366/731] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:23.538263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d749b-3e45-473d-8da3-186dabe76985 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.538811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d749b-3e45-473d-8da3-186dabe76985 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.538811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d749b-3e45-473d-8da3-186dabe76985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.556525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d28d749b-3e45-473d-8da3-186dabe76985 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:23.557741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d28d749b-3e45-473d-8da3-186dabe76985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:23.568553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14633d-6fcc-4bae-8d5a-5e9acfbb0a86 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.600703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.601874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.602308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.602721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.613791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.615450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.622945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 367/732] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:23.624499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5138-9f6a-41bd-a7c3-916a8e95f6fb None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.624835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5138-9f6a-41bd-a7c3-916a8e95f6fb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.625114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5138-9f6a-41bd-a7c3-916a8e95f6fb None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.626503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 368/733] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:23.627090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5138-9f6a-41bd-a7c3-916a8e95f6fb None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-69ab55c1-f23a-49c9-bc27-d8d3c18f157c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:23.645028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:23.646106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:23.647173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:23.647173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:23.647173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:23.647173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:23.653401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 366/734] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:23.654131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.654514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.654762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.671603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:23.672808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:23.732625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5138-9f6a-41bd-a7c3-916a8e95f6fb None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.752663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 367/735] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:23.752663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.780765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.781432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.781709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.781984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.783072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:23.783376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.784833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.802100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 369/736] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:23.807235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:23.846623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354115e4-6803-41a5-9c86-c3627c31b49b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.912540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1a73-3e01-4410-a6a3-119cb4337079 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:23.947820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:23.948492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:23.948801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:23.949074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:23.949804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:23.950149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:23.951524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:23.963639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 370/737] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:23.964500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.000309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.000986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.001263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.001535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.002239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.003589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.019950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 368/738] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:23 2022] POST /identity/v3/services => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:24.020999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.072898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628dda2-4a3e-48f8-bd47-fd67cb242e61 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.115551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ef74-2c5e-42e0-a1aa-011c482cf81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.116386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ef74-2c5e-42e0-a1aa-011c482cf81e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.116687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ef74-2c5e-42e0-a1aa-011c482cf81e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.116964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ef74-2c5e-42e0-a1aa-011c482cf81e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.128563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461ef74-2c5e-42e0-a1aa-011c482cf81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-3461ef74-2c5e-42e0-a1aa-011c482cf81e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:24.136570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:24.138193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:24.144345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:24.145274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ab6a-1428-4d56-bbad-196c85828ec5 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.145350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 371/739] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:24.145350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b3084-4b5e-447e-8272-3054fbc8c6c7 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.145350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b3084-4b5e-447e-8272-3054fbc8c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.145350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b3084-4b5e-447e-8272-3054fbc8c6c7 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.145350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 372/740] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:24.145350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b3084-4b5e-447e-8272-3054fbc8c6c7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.183125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95ab2e-a283-440f-8ba5-46e08bfff7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.183805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95ab2e-a283-440f-8ba5-46e08bfff7d4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.184141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95ab2e-a283-440f-8ba5-46e08bfff7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.184518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95ab2e-a283-440f-8ba5-46e08bfff7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.253077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b3084-4b5e-447e-8272-3054fbc8c6c7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.255737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95ab2e-a283-440f-8ba5-46e08bfff7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.257250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95ab2e-a283-440f-8ba5-46e08bfff7d4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.288255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.289118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.289397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.289670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.304900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.307412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:24.331340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:24.332282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:24.333237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:24.336959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 373/741] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:24.337978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.374553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 369/742] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 16 18:38:24.375910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.376196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.376474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.393812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:24.394975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:24.437923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc5d40-fd8d-486a-9cac-aabb4a517a07 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.458998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 370/743] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 16 18:38:24.459910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.480138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.481077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.481540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.481960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.482847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:24.483347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.485207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.493299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 374/744] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:24.494579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.568433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb0d16-ea4e-4fbf-8521-38df15975bbf None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.590513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f1e34-a8be-4cb2-8987-e3b8c58837b2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.602875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.603547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.603831 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.604114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.604976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:24.605285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.606739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.623830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.628810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 371/745] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:24.628954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.628954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.629124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.629199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.645013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.646437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:24.663251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:24.664158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:24.665681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:24.665681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:24.665681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:24.665681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:24.665681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 375/746] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:24.665681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.730416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24315a-22a4-4eb8-be24-fef063888ef9 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.760540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b787e91-5303-4cc1-84ec-33a979db23a5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.763349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.764002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.764276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.764544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.765278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.766610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.788657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 372/747] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:24 2022] POST /identity/v3/services => generated 240 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:24.788657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.801317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.802180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.802641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.802850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.815537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.817131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:24.836334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:24.837456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:24.838556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:24.838556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:24.838556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:24.838556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:24.840470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 376/748] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:24.841627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:24.933288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251ece-6091-4aa2-89ac-bd4ad7d659f1 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.950585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a4264-5808-4f3a-8bce-4f07086c2644 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:24.982626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.984104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.984611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.985114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.987104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:24.987755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:24.988035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:24.988301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:24.989066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:24.989364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:24.990741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.003169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.003563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 377/749] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:25.004977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.005169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:25.027982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:25.029088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:25.030603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:25.030603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:25.030603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:25.030603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:25.030603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 373/750] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:25.030603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.124431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727c7b0-f677-4264-82aa-e0d309c177b0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.142863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef935e4-2cf3-4f07-addf-3452b7d633a6 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.158358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.159016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.159294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.159563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.160274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:25.160572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.161965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.174274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 378/751] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:25.175152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.177976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.179053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.179561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.179984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.188802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.190618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:25.205663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:25.206754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:25.207830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:25.209517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 374/752] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:25.210445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.276518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de1f78-d792-4cfa-87f1-6ad584175bb3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.310976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.311729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.312078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.312416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.313220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:25.313668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.315030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.316227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e90a9c-4529-49c6-970a-c704ccca0ced None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.353755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.354894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.355326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.355863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.365435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.367200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:25.384134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:25.385844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:25.386999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:25.390677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 375/753] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:25.391806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.455873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 379/754] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:25.456952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.497771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d6bb2-f572-4270-a8ca-794b63da8871 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.532973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.533840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.534318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.534739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.535730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:25.536195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.537897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.550403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 376/755] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:25.551391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.555422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae225e-ec08-40c5-8ace-b236cc0b9f0e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.588178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.588870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.589147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.589415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.590232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:25.591052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.592894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.609817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 380/756] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:25.610648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.654030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de93de8-d43d-4120-8b40-45a09a92969e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.689540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.690710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.691195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.691666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.699013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.700905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.708172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2822d-f84e-4b1c-be8f-03b3cb7d5800 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:25.715942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:25.717086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:25.717976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:25.720318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 377/757] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:25.721345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.752528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.753896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.754557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.755379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.756801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=16b2497bffb3487bbd025f22463d5a79) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:25.757452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.759840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.771234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 381/758] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/role_assignments?user.id=16b2497bffb3487bbd025f22463d5a79&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:25.772790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:25.833502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9922c98-14ff-496b-adcc-5bd537308dcf None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.870791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.871445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.871721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.872548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.896915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a925c-927d-4c2c-99fa-d158a42cf791 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:25.946332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.947378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:25.947836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.948518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:25.949022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:25.949514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:25.950553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:25.951067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:25.952728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:25.984898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 382/759] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:26.000739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.059685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 378/760] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:38:26.065253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.146455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e33e0-0ef8-4b98-9275-7a8ab2fe9190 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.192689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.192689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.192689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.192689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/b163f5034b7a4d96a224c07b9c1428c3/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.193773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=b163f5034b7a4d96a224c07b9c1428c3, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.195309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.211936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 383/761] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:25 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/b163f5034b7a4d96a224c07b9c1428c3/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:38:26.295871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2badc8-847d-4398-98c7-3116faf0cef5 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.341835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.342656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.342934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.343208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.354754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.356808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-b2387e69-6fb1-4b85-a6c9-38adf856a92f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:26.387279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:26.389238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:26.390479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:26.390479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:26.390479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:26.390479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:26.390479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 379/762] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:26.397213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.444534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc429d7-8e6e-4402-905b-22a19d65a270 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.444886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc429d7-8e6e-4402-905b-22a19d65a270 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.445125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc429d7-8e6e-4402-905b-22a19d65a270 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.446746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 380/763] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:26.452263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.452545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.452861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.472466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:26.473670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:26.517167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486e998-0362-473f-a709-b29750f4dbb3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.540309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 381/764] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:26.551270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.551980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.552258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.552524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.553298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:26.553640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.555050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.562628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.567333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 384/765] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:26.573584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.663615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d40e65-2ffa-434d-8c2d-ec3317a515c4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.673553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fca6e-05b3-449e-a39b-c14efe4179a8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.696994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d330950-c993-4447-92c3-c087455996bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.697790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d330950-c993-4447-92c3-c087455996bd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.698067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d330950-c993-4447-92c3-c087455996bd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.698339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d330950-c993-4447-92c3-c087455996bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.698987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d330950-c993-4447-92c3-c087455996bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.700910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d330950-c993-4447-92c3-c087455996bd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.707658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.708444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.708756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.709033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-9d330950-c993-4447-92c3-c087455996bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:26.714572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:26.717318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:26.719310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:26.721173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.721173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.729154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 382/766] 10.222.0.36 () {56 vars in 1129 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:26.729154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d330950-c993-4447-92c3-c087455996bd None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:26.732327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:26.734371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:26.736492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 385/767] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:26.740762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.842416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f2d73-027f-44b1-9ef7-cdbe50c5cb75 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.875886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.876811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.877257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.877667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.878580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:26.879023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.880573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.893659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 386/768] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:26.899521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:26.911969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d330950-c993-4447-92c3-c087455996bd None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:26.978252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:26.978922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:26.979199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:26.979470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:26.980180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:26.980484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:26.981906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:26.993053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 383/769] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:26.998894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:27.006937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cf65d-1cf4-49a8-85c8-50aafd403ac6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:27.041750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:27.042721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.043186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.043762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.044785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=7bab459b1a3e48b0973482e9193aeb0e) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:27.045233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:27.047117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:27.053616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 387/770] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/role_assignments?user.id=7bab459b1a3e48b0973482e9193aeb0e&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:27.123061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ccb15-fb5a-4352-868b-b5db3c855cb7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:27.177465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:27.178156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.178442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.178718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.179439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:27.179746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:27.181159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:27.189363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 384/771] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 16 18:38:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:27.194841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:27.327948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dac23-218b-4564-8880-cb7c3cf1da44 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:27.362939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:27.363957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.364448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.364943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.365956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] RBAC: Adding query filter params (interface=public, service_id=662947ce712b44b8bc395c31d28423f5, region_id=RegionOne) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:27.366500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:27.368204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a238d-0326-4a34-a392-ca6d7350b05d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:27.376929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 388/772] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:38:27 2022] GET /identity/v3/endpoints?service_id=662947ce712b44b8bc395c31d28423f5&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:27.385725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:27.486068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b79c6-a12b-406f-94dd-14281d648c1a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:27.520257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857b089-d3d7-4347-802e-e3d0b7a0af85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:27.521308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857b089-d3d7-4347-802e-e3d0b7a0af85 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.521855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857b089-d3d7-4347-802e-e3d0b7a0af85 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.522329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857b089-d3d7-4347-802e-e3d0b7a0af85 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.523205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857b089-d3d7-4347-802e-e3d0b7a0af85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:27.524772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857b089-d3d7-4347-802e-e3d0b7a0af85 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:27.530964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 385/773] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:27 2022] GET /identity/v3/services => generated 772 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:27.560903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79c53c-6c6e-48c4-82c5-3468403b030c None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.561420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79c53c-6c6e-48c4-82c5-3468403b030c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.561496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79c53c-6c6e-48c4-82c5-3468403b030c None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.563139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 389/774] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:27.568807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.569440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.569911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.593039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:27.594607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:27.666569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 386/775] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:27.702001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79c53c-6c6e-48c4-82c5-3468403b030c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:27.837209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79c53c-6c6e-48c4-82c5-3468403b030c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:27.872348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:27.873089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:27.873369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:27.873639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] PATH_INFO: `/v3/services/network` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:27.874289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:27.875645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:27.887801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:27.888833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:27.889774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 390/776] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:27 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:27.894863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:28.052878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf166a4-cc90-4393-a250-9850c71afc2b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:28.097884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:28.098550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.098830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.099105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.099812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:28.100117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:28.105576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:28.110656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 387/777] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:28.116165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:28.294724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745ee95-c462-4afe-859a-70394a8ca4cc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:28.328759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:28.329417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.329685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.329951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.330646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] RBAC: Adding query filter params (type=network) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:28.330945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:28.332309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e71bda-fbe1-4d75-9e66-eebf8753c49c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:28.341141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 391/778] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:28.349764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:28.459079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6cceb-153e-4e48-8c29-f2642b91d5f8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.459427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6cceb-153e-4e48-8c29-f2642b91d5f8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.459618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6cceb-153e-4e48-8c29-f2642b91d5f8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.460975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 392/779] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:28.465999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.466281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.466547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.494772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:28.495946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:28.536450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdd70a-c638-48de-97f7-c08d09fd47c7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:28.570989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 393/780] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:28.607257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:28.613371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:28.614037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.614312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.614582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.615372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] RBAC: Adding query filter params (interface=public, service_id=f4de8a94c94f4bf09801e3d8fae8b3c1, region_id=RegionOne) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:28.615681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:28.617083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:28.631895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 388/781] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:38:28 2022] GET /identity/v3/endpoints?service_id=f4de8a94c94f4bf09801e3d8fae8b3c1&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:28.636792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:28.752237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5628f6-a140-4a83-bcc4-6c3705af6ca4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:28.822406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:28.823550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.824043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.824529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.827013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae50dd2-78c5-4e21-8609-f31cfbaf0aaa None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:28.847903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:28.862424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:28.863390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:28.864219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:28.866645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 394/782] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:28.873169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:28.903956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:28.904947 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:28.905497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:28.905967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:28.906910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:28.908742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:28.918014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 389/783] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:28 2022] GET /identity/v3/services => generated 772 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:29.046160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006601f-4e0d-4698-b9d0-289fea16d7a2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:29.096056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:29.096678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.096964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.097235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.097991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:29.098292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:29.099677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:29.109577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 395/784] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:29.115103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:29.246867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b3055-67f2-454e-8330-1e64f9ff963e None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:29.296028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:29.296848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.297110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.297381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.317229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:29.318700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f185b897-38db-404c-9fb0-f3330178dc36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:29.339766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:29.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:29.342144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:29.342144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:29.342144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:29.342144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:29.346215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 390/785] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:29.347734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:29.380755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4feb-34cf-4a7c-af80-625c097e3103 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.381368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4feb-34cf-4a7c-af80-625c097e3103 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.381843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4feb-34cf-4a7c-af80-625c097e3103 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.383629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 391/786] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:29.389346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.389855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.390330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.408272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:29.409804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:29.485020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9ddf2-4c12-47c0-8478-55690ad0ff5e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:29.511219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 392/787] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:29.526235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:29.526909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.527190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.527463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.528880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:29.529190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:29.530584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:29.538264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:29.552430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 396/788] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:29.557948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:29.712977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d29a39-9701-4cbd-8bda-4d53208c0039 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:29.725961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11e78f-20c1-4697-b36d-283574aba47a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:29.756735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641620a6-e6d1-4188-aed9-0bf00dd4542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:29.757527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641620a6-e6d1-4188-aed9-0bf00dd4542e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.757804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641620a6-e6d1-4188-aed9-0bf00dd4542e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.758075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641620a6-e6d1-4188-aed9-0bf00dd4542e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.767608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:29.768689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.768986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.769254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641620a6-e6d1-4188-aed9-0bf00dd4542e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:29.769357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.776525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-641620a6-e6d1-4188-aed9-0bf00dd4542e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:29.777279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:29.778301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:29.779149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:29.779777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:29.779847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 393/789] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:29.780370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c49d-6af5-495e-8f1d-b4e6d7c5afba None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.781534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c49d-6af5-495e-8f1d-b4e6d7c5afba None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.781534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c49d-6af5-495e-8f1d-b4e6d7c5afba None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.783998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 394/790] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:29.787025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c49d-6af5-495e-8f1d-b4e6d7c5afba None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e22774de-dd31-4e62-9e9b-fd8d5f585396 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:29.798747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:29.799725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:29.800684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 397/791] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:29.802165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.802516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.802871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.824446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:29.826318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:29.897629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c49d-6af5-495e-8f1d-b4e6d7c5afba None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:29.916712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 398/792] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:29.917910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:29.958589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:29.959591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:29.960066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:29.960314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:29.961098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:29.961403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:29.962773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:29.973237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 395/793] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:29.973990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.060330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9d0cb-354e-4481-8f87-29ddc108de22 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.080995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1059f-98fb-4399-98cc-93f108219b75 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.104388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.105077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.105354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.105625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.106479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:30.106789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.108207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.123105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.124405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.124908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.125361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.125982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 399/794] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:30.126905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.139479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:30.150193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.151221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:30.152058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:30.159501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 396/795] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.159501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.259508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4157b-7b08-4d09-b41c-724300f10f33 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.265582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c862c-392a-4c54-a130-cf93bdd61ae4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.310589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.311508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.311993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.312497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.313016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.314040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.314386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.314734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.324515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.326242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.326584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.328369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:30.343994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.344963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.345839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:30.345839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:30.345839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:30.345839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:30.345839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 397/796] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.346537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-8c9ba291-5c93-412f-8fe0-d0b2e754bdc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:30.352662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.353541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 400/797] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631573-fda4-4a15-835b-c8b47f237ebf None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631573-fda4-4a15-835b-c8b47f237ebf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16631573-fda4-4a15-835b-c8b47f237ebf None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.355187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 401/798] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:30.355791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16631573-fda4-4a15-835b-c8b47f237ebf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.445047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4e9e1-2b48-4500-b0df-1d6a06bd48b0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.461116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16631573-fda4-4a15-835b-c8b47f237ebf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.476915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.477829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.478270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.478548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.479291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:30.479597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.481540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.493932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 398/799] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:30.493932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.499075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.499726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.500004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.500272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.501015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:30.501314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.502656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a420565-3370-4d16-8172-d8d9bf9020fb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.513744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 402/800] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:30.514897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.515183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.515457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.532954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:30.534388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:30.589524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942ac1-9f8f-4fbc-8a79-36046b9e5727 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.600951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 403/801] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:30.602076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.630154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.630816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.631096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.631365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.632075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:30.632380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.633762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.645223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 399/802] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:30.645998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.706143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49138c44-0da8-41e5-a9b0-bf9ac51d8e65 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.739425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.740209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.740481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.740792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.744110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace6391-d089-4cea-adfd-414f80aeb045 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.750634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.752076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:30.769241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.770147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.771063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:30.771063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:30.771063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:30.771063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:30.775128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 404/803] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.775717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.778899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.779679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.779955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.780226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.787099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.788510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:30.802509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.803571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 400/804] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.804448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.884463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115d675-23a5-4de7-a497-d628319bddf5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.899935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557067f-012c-4072-b60c-5f03d1b50001 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:30.923778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.924849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.925284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.925556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.926212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.927561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.933019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:30.933793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:30.934094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:30.934367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.939386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:30.941733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 405/805] 10.222.0.36 () {56 vars in 1129 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.942579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:30.947070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:30.948516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:30.969160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:30.970084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 401/806] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:30.976085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.043758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113655af-842c-4e8e-8018-21eff84e8cc2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.073184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f777c53-cdc5-4c9f-96b0-92eb58ff7926 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.076000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.076679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.076964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.077239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.077953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:31.078251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.079625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.092121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 406/807] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:31.092954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.113434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.114091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.114377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.114649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.115435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:31.115767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.117236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.129129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 402/808] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:31.129762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.193650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f0ff1-6cf8-44c8-8217-fb22c7ef3579 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.231186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.231859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.232137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.232409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.233157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:31.233551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.234855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.235425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a322-7488-4fcf-a7d9-822703a67233 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.244881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 407/809] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:31.244881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.276471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.277162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.277443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.277721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.278584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:31.278889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.280309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.298449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 403/810] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:31.299168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.343972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddee8a-a9c3-476e-a7ad-2295d99a8cc7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.376970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.377771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.378047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.378318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.385294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.386724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:31.401913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:31.403283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 408/811] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:31.404373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.408997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ab936-9943-411d-a559-1a63a6e553f7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.441018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.441889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.442167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.442440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/16b2497bffb3487bbd025f22463d5a79/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.444370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=16b2497bffb3487bbd025f22463d5a79, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.445823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.460808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 404/812] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:31 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/16b2497bffb3487bbd025f22463d5a79/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:31.461610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.516415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48e89-55eb-440d-8b36-2291fa3024b3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.553293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.553963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.554250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.554522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.556340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:31.556340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.557000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a663-c7c2-4b70-b0ca-bfc9887db9d6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.571559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 409/813] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:31.571764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cfa8a-f43e-44bc-8044-50ddf723cd34 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.571886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd71b3-5bb1-437b-9e11-c6bccca6e880 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.571886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd71b3-5bb1-437b-9e11-c6bccca6e880 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.571886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd71b3-5bb1-437b-9e11-c6bccca6e880 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.571886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 410/814] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:31.571886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd71b3-5bb1-437b-9e11-c6bccca6e880 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.600235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd86fc-82b8-4317-8570-c22a2f1e752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.601016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd86fc-82b8-4317-8570-c22a2f1e752a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.601218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd86fc-82b8-4317-8570-c22a2f1e752a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.601519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd86fc-82b8-4317-8570-c22a2f1e752a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.671126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd86fc-82b8-4317-8570-c22a2f1e752a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.672714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd86fc-82b8-4317-8570-c22a2f1e752a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.675910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd71b3-5bb1-437b-9e11-c6bccca6e880 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.708831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.709489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.709795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.710075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.710901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:31.711206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.712641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.733996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 411/815] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:31.733996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.770342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 405/816] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:38:31.771798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.772155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.772503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.788567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:31.789786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:31.835703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e22e-bdf3-45f7-a7f6-bd1b12cdf623 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:31.857768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 406/817] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:31.858614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.882786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:31.883442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:31.883723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:31.883998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:31.884839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:31.886180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:31.916658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 412/818] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:31.916658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:31.969571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35edad-1a29-4349-8958-73b1986914eb None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.018488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.019291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.019574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.019848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.033201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.034730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.036082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250b12-79f5-4c3d-8697-6ee27a042883 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:32.054393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:32.055737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:32.056761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:32.056761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:32.056761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:32.056761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:32.056761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 407/819] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:32.057136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.076828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.077892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.078388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.078838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.091210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.093050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:32.111452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:32.112401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:32.113632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:32.113632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:32.113632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:32.113632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:32.115725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 413/820] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:32.116745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.172684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375c84e-4541-4908-95ca-875b1e530393 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.201619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.202408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.202682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.202948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] PATH_INFO: `/v3/services/network` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.203594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.204981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:32.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:32.218414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:32.219373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 408/821] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:32.220183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.249765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017f049-a8c7-44fa-99c2-df3455cbb223 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.290295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.290949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.291258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.291540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.292250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:32.292546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.294518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.311205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 414/822] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:32.312044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.339661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfba86-9d25-4370-b310-f306ec1e8881 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.388362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.389382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.389664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.389935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.390671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:32.390973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.392324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.404523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 409/823] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:32.405459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.423790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4876af-fd05-491e-95ae-899d83d17ffb None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.467631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.468828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.469739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.470197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.471353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] RBAC: Adding query filter params (name=network) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:32.471852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.474297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.480242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 415/824] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:32.481450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.507011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd99c10-e99f-4ba3-ba11-f47381722f2a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.541533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.542637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.542749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.543100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.550093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.551554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:32.565295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:32.566197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:32.567075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:32.571267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 410/825] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:32.572251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.604924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158e97-7a7c-4711-8ccf-88a4ab97eb35 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.647795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.649295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.649805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.650254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.660027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.661809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.675092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3dbd-190f-42c8-8b44-117338cf7cee None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:32.679562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:32.680440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:32.681351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:32.684725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 416/826] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:32.685563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.716962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.717636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.717913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.718187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.718893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] RBAC: Adding query filter params (type=network) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:32.719258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.720706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.726036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 411/827] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:32.726830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.792624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d7a26-9efa-4e5a-9f84-3dc898444fc1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.826650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.827331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.827607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.827877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.828659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:32.828992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.829162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524fc9c-99c4-4e43-abb5-70deb12e0fe3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.830419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.847854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 417/828] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:32.848667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.863745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.864452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.864777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.865054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.865835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:32.866141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:32.885978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 412/829] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:32.886807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:32.946435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115f708-6078-4ccc-afb6-68ddc09ca258 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:32.985182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:32.985822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:32.986096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:32.986362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:32.987079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:32.988430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:33.001497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d34d1-e132-47a3-aceb-cf891c4fa5ed None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:33.018509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 418/830] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:33.018509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:33.076700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758a810-5321-44a9-a55e-96153ec17328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:33.077827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758a810-5321-44a9-a55e-96153ec17328 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:33.078102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758a810-5321-44a9-a55e-96153ec17328 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:33.078373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758a810-5321-44a9-a55e-96153ec17328 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/7bab459b1a3e48b0973482e9193aeb0e/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:33.112661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758a810-5321-44a9-a55e-96153ec17328 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=7bab459b1a3e48b0973482e9193aeb0e, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:33.114117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758a810-5321-44a9-a55e-96153ec17328 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:33.136764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 413/831] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:32 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/7bab459b1a3e48b0973482e9193aeb0e/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:33.198158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc384b-3564-49c9-bbf5-2f876b92e4b0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:33.248739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:33.249434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:33.249708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:33.249979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:33.250855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=b163f5034b7a4d96a224c07b9c1428c3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:33.251166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:33.252712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020b8f3-7f65-4b02-a2cd-293c4f138c13 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:33.266004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 419/832] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:33 2022] GET /identity/v3/role_assignments?user.id=b163f5034b7a4d96a224c07b9c1428c3&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:33.759034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ddf36-581d-4566-9477-e4b13defc0c0 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:33.759562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ddf36-581d-4566-9477-e4b13defc0c0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:33.759562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ddf36-581d-4566-9477-e4b13defc0c0 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:33.761220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 414/833] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:33.767057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:33.767347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:33.767625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:33.785161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:33.786334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:33.860744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 420/834] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:33.878291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ddf36-581d-4566-9477-e4b13defc0c0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:33.981701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ddf36-581d-4566-9477-e4b13defc0c0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.015492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.016203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.016485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.016789 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.028438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:34.036332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:34.037401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:38:34.038227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:34.042499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 415/835] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:34.044060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.146309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e221a98-d60c-47a4-afb3-902bca2dece4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.181651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.182318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.182602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.182875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.183614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:34.183944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.185330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962bf8f-bdea-472d-9d94-aca95c0639aa None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.198110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 421/836] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:34.199424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.299156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253345d0-563b-4977-93a6-9e504ebf27bd None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.314862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19f8a-226f-43c4-8dc0-efc7e2d31cb2 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.315203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19f8a-226f-43c4-8dc0-efc7e2d31cb2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.315466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b19f8a-226f-43c4-8dc0-efc7e2d31cb2 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.317076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 422/837] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:34.322441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.322725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.323003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.334230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.335020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.335297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.335569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.340314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:34.341695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:34.347064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.348518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:34.366517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:34.368662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:34.370691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:34.370691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:34.370691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:34.370691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:34.370691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 416/838] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:34.375893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.422347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 423/839] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:34.444715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.488863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbd897-745c-4dd5-a769-60164f4dea79 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.532260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.533598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.533876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.534147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.534864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:34.535523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.537390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.556801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 417/840] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:34.558384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.563129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7216a2-114c-4756-98e4-4045b2763cfd None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.597338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.598217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.598498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.598765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.599406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.600788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:34.612875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:34.613961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:34.615137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:34.618953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 424/841] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:34.619883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.683094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff1adb-8742-4de6-9528-e4c4e25f0a95 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.716416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.717245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.717518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.717785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.727149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.728847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.732504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e126-14a3-4bcc-beb4-a0a9f27a9cf1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:34.751696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:34.752646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 16 18:38:34.753586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:34.756712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 418/842] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:34.760706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.773308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.774094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.774470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.774842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.775868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:34.775984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.777487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.787066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 425/843] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:34.788306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.865187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1dac9-4f8f-403b-8fd3-29b2aa9d7727 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.887885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63dfe-d1ac-4b5f-a9ae-25c5ff506562 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:34.899642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.900608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.901115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.901585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.902616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:34.903157 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.904811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.922494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:34.923160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:34.923436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:34.923766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:34.924442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 419/844] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:34.928852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:34.929388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:34.929963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:34.931159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:34.945289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 426/845] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:35.046368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148f92-ae23-42ed-881d-d497b98ede3b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:35.081788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:35.082749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.083246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.083710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.155269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:35.156664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:35.255563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 420/846] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:38:35.261029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.370229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824c0e-bfeb-417c-ac35-2ee2e9c500b3 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:35.405371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:35.406080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.406356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.406625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.417569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:35.419529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-958bb188-aefa-4dd4-a179-a898349579a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:35.436683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:35.437616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:35.438512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:35.438512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:35.438512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:35.438512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:35.441738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 427/847] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:35.444059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.502627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73622a4d-f12d-4c9e-a412-52d6d30e04fb None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.503201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73622a4d-f12d-4c9e-a412-52d6d30e04fb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.503690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73622a4d-f12d-4c9e-a412-52d6d30e04fb None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.505624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 428/848] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:35.513746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.513938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.514143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.531342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:35.532909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:35.562559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32ded-dc1f-4d0d-a3de-ede2d8a7fe7f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:35.599555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:35.600600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.601333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.601800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.602708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:35.603220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:35.604816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:35.604990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 429/849] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:35.620486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 421/850] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:35.622454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.623796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.718430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22768061-d55d-4305-a1ca-20dbcb0bd7c3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:35.719455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765dc78-2981-4d6b-8818-df766067886b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:35.750748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fba253-0bd2-4cdb-b2bc-b5d800e7907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:35.751533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fba253-0bd2-4cdb-b2bc-b5d800e7907d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.751728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:35.751860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fba253-0bd2-4cdb-b2bc-b5d800e7907d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.752065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fba253-0bd2-4cdb-b2bc-b5d800e7907d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.752431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.752738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.753016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.759582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:35.761046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:35.762518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fba253-0bd2-4cdb-b2bc-b5d800e7907d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-f1fba253-0bd2-4cdb-b2bc-b5d800e7907d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:35.770194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:35.771180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:38:35.772014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e53484e8-f191-48bb-989e-5686508e1710 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:35.775203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:35.776607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:35.777520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:35.778463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 430/851] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:35.778463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740e86-15c8-4aea-b3fb-9527adff2730 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.778463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740e86-15c8-4aea-b3fb-9527adff2730 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.778463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740e86-15c8-4aea-b3fb-9527adff2730 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.784043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 431/852] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:35.784043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740e86-15c8-4aea-b3fb-9527adff2730 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.784401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 422/853] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:35.785270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.785553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.785828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.802363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:35.803532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:35.872691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 423/854] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:35.872691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.879632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740e86-15c8-4aea-b3fb-9527adff2730 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:35.913439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:35.914079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:35.914368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:35.914637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:35.915367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:35.915668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:35.917070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:35.930250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 432/855] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:35.930250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:35.981876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c03eb-3201-4c26-9917-5daaad7850c9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.014406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.015176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.015349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.015616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.016382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:36.016714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.018123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.024032 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ec8f2-ef6e-4d72-92d1-42be968e7777 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.033233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 424/856] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:36.034469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.059139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.059925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.060197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.060463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.061146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.062491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:36.074343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:36.075280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:36.076130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 433/857] 10.222.0.36 () {56 vars in 1132 bytes} [Mon May 16 18:38:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:36.077023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.134034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562651f3-b15a-40f5-aa74-baf828441122 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.168486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.169289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.169565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.169833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.178195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39363739-bbb4-4561-ab73-8155b84168f2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.179823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.181277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:36.197681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:36.198570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:36.199452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:36.199452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:36.199452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:36.199452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:36.199452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 425/858] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:36.200253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.216740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.217387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.217665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.217943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.218751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=16b2497bffb3487bbd025f22463d5a79) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:36.219063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.220471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.226154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 434/859] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/role_assignments?user.id=16b2497bffb3487bbd025f22463d5a79&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:36.226931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.321981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c4a73-8f77-4185-9498-825c104323ef None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.354675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920fce8-69df-4b83-b442-f0bae133de57 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.365927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.367181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.367643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.368260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.369695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:36.370324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.372556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.379341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 426/860] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:36.387890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.402305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.403826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.404176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.404525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.405416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:36.405773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.407254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.425243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 435/861] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:36.426262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.503374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdded3f-630c-4dbf-80fd-54abd25fa5d3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.538893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f7b50-d07a-4448-be9d-86c4c87309cc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.539234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.539902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.540338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.540690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.542291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:36.542603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.544057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.550065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 427/862] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:36.580355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.581144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.581429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.581704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.590299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.591896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:36.609794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:36.610777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:36.612145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 436/863] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:36.617150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.738539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c362e2d-2048-4e13-9361-675ed2644355 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.775980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.777455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.777801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.778084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:36.778940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:36.779250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:36.780716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:36.800045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 428/864] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:36.805586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:36.908830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd5edf-5026-4be1-a491-8777ce2e19d0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:36.944039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa2f4b-1b4d-4a2d-b4c0-971d68dc7ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:36.945534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa2f4b-1b4d-4a2d-b4c0-971d68dc7ec5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:36.946010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa2f4b-1b4d-4a2d-b4c0-971d68dc7ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:36.946477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa2f4b-1b4d-4a2d-b4c0-971d68dc7ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.014047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa2f4b-1b4d-4a2d-b4c0-971d68dc7ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.015492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa2f4b-1b4d-4a2d-b4c0-971d68dc7ec5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:37.123914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 437/865] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:38:37.125360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:37.222340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7f6e0-7f9a-4f05-ab98-0b9d5e95d121 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:37.257049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:37.257841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.258122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.258397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.268528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.269993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:37.278062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b0e27-5ea5-4cbb-ab26-8b53a0da9567 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.278360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b0e27-5ea5-4cbb-ab26-8b53a0da9567 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.278633 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b0e27-5ea5-4cbb-ab26-8b53a0da9567 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.279987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 438/866] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:37.285349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.285634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.285911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:37.290874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:37.291723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:37.293075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:37.293075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:37.293075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:37.293075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:37.293075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 429/867] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:37.301586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:37.308186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:37.309454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:37.394943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 439/868] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:37.400293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de99a17-1789-4e50-893a-1fa3ddc6485b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:37.417912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:37.443626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:37.444248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.444529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.444831 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.445546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:37.445852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.447246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:37.463302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 430/869] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:37.467716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:37.535817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b2d80-b337-4ba3-bc13-94dd300afbfe None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:37.571602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:37.572532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.572998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.573429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.574292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.575832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:37.590550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f14eb-d61a-4d45-8fe5-5ac045dc0161 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:37.601228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 440/870] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:37 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:38:37.637500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:37.638477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.638751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.639017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.647541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.649208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:37.666329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:37.667299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:37.668478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 431/871] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:37.675353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:37.787947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24754360-4333-436e-aea0-34d45ece0e1b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:37.822345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:37.823375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.823868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.824337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.825954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:37.826460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.828094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b468-287b-4139-9f9b-d5ac740baae4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:37.844859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 441/872] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:37.849053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:37.958055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb08c93-4c02-40ac-88cd-e91ac3f59088 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:37.993085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:37.993729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:37.994016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:37.994292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:37.995112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=7bab459b1a3e48b0973482e9193aeb0e) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:37.995419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:37.997425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afc8dc-595a-486f-8703-9ba76eef545d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:38.008769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 432/873] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:37 2022] GET /identity/v3/role_assignments?user.id=7bab459b1a3e48b0973482e9193aeb0e&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:38.630386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566ff4-0c40-4f27-a634-bbd2374b7867 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:38.630914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566ff4-0c40-4f27-a634-bbd2374b7867 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:38.630992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566ff4-0c40-4f27-a634-bbd2374b7867 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:38.632370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 442/874] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:38.637699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:38.637988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:38.638265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:38.654945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:38.656123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:38.722707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 433/875] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:38.745232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9566ff4-0c40-4f27-a634-bbd2374b7867 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:38.856621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9566ff4-0c40-4f27-a634-bbd2374b7867 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:38.890066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:38.891194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:38.891669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:38.892151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:38.893050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:38.894602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-a18bfbbd-71ac-4ac5-ab3d-14e780b9450c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:38.913081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:38.914123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:38.914945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:38.917824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 443/876] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:38.925453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:38.929053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb083b8-bace-40a5-b620-23d302ccc74f None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:38.929343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb083b8-bace-40a5-b620-23d302ccc74f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:38.929619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb083b8-bace-40a5-b620-23d302ccc74f None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:38.931228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 444/877] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:38.936259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:38.936544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:38.936862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:38.957869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:38.957869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:39.032966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 445/878] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:39.040592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801277bb-36f7-4045-b027-95d9dbb559f0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:39.047410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:39.082598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:39.083257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:39.083537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:39.083807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:39.084509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:39.084836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:39.086191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:39.093880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 434/879] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:39.099623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:39.164729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9f13d-1a9c-4c55-b91d-f5f65ba16fe0 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:39.201280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a336d-1fa2-404c-b23f-dafc5fabf6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:39.202304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a336d-1fa2-404c-b23f-dafc5fabf6fb None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:39.202792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a336d-1fa2-404c-b23f-dafc5fabf6fb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:39.203297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a336d-1fa2-404c-b23f-dafc5fabf6fb None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:39.204254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a336d-1fa2-404c-b23f-dafc5fabf6fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:39.205936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a336d-1fa2-404c-b23f-dafc5fabf6fb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:39.206308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb75f7-e091-46ec-954c-e1bad69fd4cb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:39.224130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 446/880] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:39 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:38:39.268242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:39.269447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:39.269928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:39.270394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:39.271346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:39.271846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:39.273627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4d327-5521-426f-8773-45faf9cb7eb5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:39.280178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 435/881] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:39.982995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64323b-6d89-449c-a8e1-79a35253795e None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:39.983513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64323b-6d89-449c-a8e1-79a35253795e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:39.983589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64323b-6d89-449c-a8e1-79a35253795e None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:39.985195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 447/882] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:39.990168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:39.990594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:39.990928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.008248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:40.009463 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:40.079599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 436/883] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:40.101282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64323b-6d89-449c-a8e1-79a35253795e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.215014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64323b-6d89-449c-a8e1-79a35253795e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.248951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.249746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.250020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.250292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.250936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:40.252399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-5faa94af-7d9e-4953-ae9d-b3d187448f9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:40.264106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:40.265119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:40.265965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 448/884] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:40.271465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.349865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb7895-7da7-42f2-a06b-d04556b9347b None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.350271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb7895-7da7-42f2-a06b-d04556b9347b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.350435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb7895-7da7-42f2-a06b-d04556b9347b None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.352729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 449/885] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:40.357203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.357487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.357764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.377158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:40.378469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:40.379222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192b545-75d6-43ab-a47d-6762be840b25 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.412404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.413192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.413539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.413880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.414653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:40.415026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:40.416446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:40.423618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 437/886] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:40.429506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.463288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 450/887] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:40.486522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.559587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac30521-b26b-4e4d-ad97-284ce9f1d01d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.593980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.594639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.594914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.595184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.595878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:40.596176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:40.597566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:40.599948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d106a-31bf-43ce-a216-9ac0a4e54658 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.607234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 438/888] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:40.608292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.636397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.638295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.638589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.638866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.652391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:40.662923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:40.664154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:40.665674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 451/889] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:40.671267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.712363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f780c-2c08-499f-a3ea-abf9bbe78a2b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.746014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.747532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.748023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.748494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.749506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] RBAC: Adding query filter params (interface=public, service_id=10023ea3aa8945bc9727de735a1cf5e9, region_id=RegionOne) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:40.750004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:40.751621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:40.761608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 439/890] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/endpoints?service_id=10023ea3aa8945bc9727de735a1cf5e9&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:40.762819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.798130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed444d6-950f-4d26-a22b-5fa759fec575 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.839092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.840177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.840461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.840763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.841503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:40.841805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:40.843527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:40.856707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 452/891] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:40.860026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:40.876365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b314f3a-0084-4bd8-9b37-533c51872004 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:40.916903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:40.917588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:40.918793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:40.919070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:40.919751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:40.921596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:40.936668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 440/892] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:38:40.975526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59354570-a785-407c-b63f-b4668b5b8c33 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:41.013778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:41.014540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.014838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.015111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.027152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:41.029260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:41.049517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:41.050494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:41.051803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:41.051803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:41.051803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:41.051803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:41.055606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 453/893] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:41.057068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:41.162386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766135ff-2c83-4b12-8ee6-f987fe58adc0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:41.196327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:41.197017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.197297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.197569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.198281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:41.198584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:41.200001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:41.216301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 441/894] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:41.217788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:41.321360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0ef8-d47d-459a-b376-06c075374df0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:41.355434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:41.356857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.357164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.357436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.365076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:41.366520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:41.381369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:41.382369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:41.383304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:41.387866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 454/895] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:38:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:41.389227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:41.498319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d59f0-4f2b-432b-8450-dff68d5993b4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:41.535983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:41.537046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.537524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.538007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.539034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:41.539551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:41.541196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033761f0-0bfe-4ca7-846a-b1a7b8d91ef3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:41.559473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 442/896] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 16 18:38:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:41.566015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:41.627693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f19c-a0cb-4222-acdf-15b81ab8400d None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.627883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f19c-a0cb-4222-acdf-15b81ab8400d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.628164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f19c-a0cb-4222-acdf-15b81ab8400d None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.629804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 443/897] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:41.634622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11430789-2a17-4d57-bd53-ae2747ea1060 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.634912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11430789-2a17-4d57-bd53-ae2747ea1060 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.635188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11430789-2a17-4d57-bd53-ae2747ea1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.652141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-11430789-2a17-4d57-bd53-ae2747ea1060 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:41.654062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-11430789-2a17-4d57-bd53-ae2747ea1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:41.675446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3f748-7a51-420b-91b0-660e707fc44b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:41.710910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:41.711552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.711832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.712111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.720061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 444/898] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:41.721764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39665af2-81a2-48c0-8bd9-d43c0ef21f8a None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.722238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39665af2-81a2-48c0-8bd9-d43c0ef21f8a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.722510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39665af2-81a2-48c0-8bd9-d43c0ef21f8a None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.724678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 445/899] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:41.729206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.729415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.729686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.751421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:41.752588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:41.797906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:41.799356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:41.818686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 446/900] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:41.819907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:41.914488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 455/901] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:41 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:38:41.915281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:41.918623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd683510-4315-488b-8c8d-9f38ef1ef56b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:41.959557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:41.960398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:41.960700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:41.961019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:41.961926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:41.963309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:41.975487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:41.976449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:41.977352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 447/902] 10.222.0.36 () {56 vars in 1132 bytes} [Mon May 16 18:38:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:41.978216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.022454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c57515-90a2-4302-9fb5-18583cba7a0f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.055716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.056753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.057033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.057301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.058066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.060076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.073137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 456/903] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:41 2022] POST /identity/v3/services => generated 241 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:42.074255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.076266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987625a8-53dc-4122-9e23-0735637d4c60 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.120734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.122097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.122377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.122646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.134569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.136048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:42.155948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:42.156998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:42.158149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:42.158149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:42.158149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:42.158149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:42.162618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 448/904] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:42.164958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.193978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2019-c695-4032-9bc8-e703085d7440 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.233987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.235245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.235927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.236398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.237979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:42.238513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.240108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.253569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 457/905] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:42.254865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.286725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423604a4-ee9e-4284-8b45-71d09b5a224f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.324966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.325630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.325915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.326190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.326912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:42.327215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.328578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.342456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 449/906] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:42.347676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.382417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90017e-19e3-4094-bb82-3b20d627ae03 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.421941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.423381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.424141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.424614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.425707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:42.427002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.428557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.435882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 458/907] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:42.440979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.469422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f685e8-81e4-44bd-8ea6-ff1079cd6fd9 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.504501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.505694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.506181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.506647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.513693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.515469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:42.530026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:42.530933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:42.531830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:42.538986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 450/908] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:42.540431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.540664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041d4f-3cb5-4c44-a6a5-0a526a397b53 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.574502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c0766-090a-4def-b617-face772ab92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.575236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c0766-090a-4def-b617-face772ab92a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.575425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c0766-090a-4def-b617-face772ab92a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.575687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c0766-090a-4def-b617-face772ab92a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.576480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c0766-090a-4def-b617-face772ab92a None admin] RBAC: Adding query filter params (interface=public, service_id=2ec6a0eadd7747fc92b4b75175604255, region_id=RegionOne) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:42.577327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c0766-090a-4def-b617-face772ab92a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.578181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c0766-090a-4def-b617-face772ab92a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.587779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 459/909] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/endpoints?service_id=2ec6a0eadd7747fc92b4b75175604255&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:42.588839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c0766-090a-4def-b617-face772ab92a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.642661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bbb62-5108-44fc-8e3b-27a4b610f68d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.677477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.678400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.678817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.679238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.680161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:42.680615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.682213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.687851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c0766-090a-4def-b617-face772ab92a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.699773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 451/910] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:42.701269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:42.721684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79001b2d-0531-4e6d-94a6-3a77e61ede5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.722341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001b2d-0531-4e6d-94a6-3a77e61ede5f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.722616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001b2d-0531-4e6d-94a6-3a77e61ede5f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.722885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001b2d-0531-4e6d-94a6-3a77e61ede5f None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.723553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79001b2d-0531-4e6d-94a6-3a77e61ede5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.724930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79001b2d-0531-4e6d-94a6-3a77e61ede5f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.736773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 460/911] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/services => generated 1497 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:38:42.811168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03469d8a-af19-4fa4-97f8-a3d104950185 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:42.849175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:42.850116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:42.850537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:42.850967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:42.851951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] RBAC: Adding query filter params (role.id=d9dde88ec4e44b569a261194c866b6e7, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=7bab459b1a3e48b0973482e9193aeb0e) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:42.852406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:42.854115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df648ab3-2f17-46fb-8d24-6b3e9f72f7af None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:42.867375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 452/912] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:42 2022] GET /identity/v3/role_assignments?user.id=7bab459b1a3e48b0973482e9193aeb0e&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=d9dde88ec4e44b569a261194c866b6e7 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:43.296602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048f042-9539-4b05-b93d-b6f6e2b2b7b9 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:43.297191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048f042-9539-4b05-b93d-b6f6e2b2b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:43.297191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048f042-9539-4b05-b93d-b6f6e2b2b7b9 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:43.298686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 461/913] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:43.304088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:43.304898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:43.305389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:43.323175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:43.324956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:43.393385 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 453/914] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:43.416188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048f042-9539-4b05-b93d-b6f6e2b2b7b9 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:43.519418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048f042-9539-4b05-b93d-b6f6e2b2b7b9 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:43.552653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:43.553374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:43.553645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:43.553965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:43.554610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:43.555953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:43.568063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:43.569091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:43.569950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 462/915] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:43.577753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:43.676920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74dda58-ace2-4478-b6fa-df4cd8c2b3d4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:43.710935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:43.711966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:43.712444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:43.712954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:43.713871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:43.714372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:43.715981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:43.721271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 454/916] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:43.727018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:43.825429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242705-dc7f-4917-8596-3d41942baeea None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:43.864981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:43.866503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:43.866998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:43.867470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:43.868402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:43.869163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:43.872086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a64a-38e2-4e34-88d5-cfb657f9acd5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:43.879361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 463/917] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 16 18:38:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:43.885125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:43.991209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1a16-3eb2-4add-892f-ffdb239f35d4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:44.028428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089e3f-27c9-4988-a6da-719c65c1afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:44.029555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089e3f-27c9-4988-a6da-719c65c1afdb None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:44.030036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089e3f-27c9-4988-a6da-719c65c1afdb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:44.030531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089e3f-27c9-4988-a6da-719c65c1afdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:44.031484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089e3f-27c9-4988-a6da-719c65c1afdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:44.033324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089e3f-27c9-4988-a6da-719c65c1afdb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:44.056137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 455/918] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:44.439077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9a63-b3c7-4d53-8668-828f38bcc867 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:44.439611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9a63-b3c7-4d53-8668-828f38bcc867 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:44.439611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9a63-b3c7-4d53-8668-828f38bcc867 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:44.441261 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 464/919] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:44.446795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:44.447209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:44.447377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:44.466570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:44.467750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:44.548064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 456/920] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:44.572601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9a63-b3c7-4d53-8668-828f38bcc867 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:44.678610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9a63-b3c7-4d53-8668-828f38bcc867 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:44.713117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:44.713893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:44.714169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:44.714443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:44.715696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:44.717154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:44.729462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:44.730542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:44.731361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 465/921] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:44.738716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:44.838860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e2e9a-9978-4c9f-9e79-437b85e1fbed None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:44.882662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:44.883703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:44.884210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:44.884754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:44.885721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:44.886230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:44.888448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:44.894189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 457/922] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:44.900021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.014805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf3e2-057d-4fed-b1db-5f445d023e9c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.048589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.049313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.049599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.049879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.050645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:45.050958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.052351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509cc5-58ab-451c-9afc-b548997e4c50 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.057730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 466/923] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:45.063884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.116140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8213c-5fbe-40be-9f2d-5dba370d391e None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.116470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8213c-5fbe-40be-9f2d-5dba370d391e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.116732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8213c-5fbe-40be-9f2d-5dba370d391e None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.118205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 467/924] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:45.123855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c21687-91a0-4298-9249-e7c9e446b264 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.124137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c21687-91a0-4298-9249-e7c9e446b264 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.124407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c21687-91a0-4298-9249-e7c9e446b264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.143271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f9c21687-91a0-4298-9249-e7c9e446b264 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:45.144432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f9c21687-91a0-4298-9249-e7c9e446b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:45.179649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a454dc-1030-40ff-84ce-8185217fce31 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.215794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.216443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.216754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.216972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 468/925] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:45.217075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.217801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] RBAC: Adding query filter params (interface=public, service_id=80459b42c89c4a7c859ff21eb5c8bd6b, region_id=RegionOne) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:45.218107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.219108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4853-a615-400f-9624-c430d77e4ab8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.219456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.219808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4853-a615-400f-9624-c430d77e4ab8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.220272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4853-a615-400f-9624-c430d77e4ab8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.221794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 469/926] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:45.227068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.227348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.227621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.232674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 458/927] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/endpoints?service_id=80459b42c89c4a7c859ff21eb5c8bd6b&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:45.237808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.249298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:45.250467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:45.319199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 470/928] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:45.320036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.349216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72916-6ac9-414a-9b2e-9db0f39ee13a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.382839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.383497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.383777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.384053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.384784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.386181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.396823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 459/929] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:38:45.398119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.432835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b5cf-3fa8-45a0-9409-625f51415e8c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.477221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.478071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.478341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.478610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.479649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.481867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:45.497827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:45.498832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 16 18:38:45.499674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:45.504776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 471/930] 10.222.0.36 () {56 vars in 1132 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:45.505728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.532508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b210f53-5598-40bd-970a-517f0d51c5db None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.586728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.587945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.588376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.588735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.602651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:45.611616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:45.612583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:45.613466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 460/931] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:45.620360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.629065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c139e-9e9b-4935-9a4b-388ae02df17d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.662768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.663658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.663941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.664214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.664978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:45.665258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.666679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.672490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 472/932] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:45.678884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.742814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865e467-d7e3-4a5b-bea1-a57cfec15398 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.779037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.779845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.780219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.780588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.781450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:45.781850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.783552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.797255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 461/933] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:45.798207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.805847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c1437-996b-4601-80b7-dba231ef3163 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.838507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.839087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.839364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.839633 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.840902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:45.841206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:45.842666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:45.852253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 473/934] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:45.853362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:45.937124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41a1be-b0f7-4fe3-8041-d09d1d2de846 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.968766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d2c90-f46f-4e44-ab65-bf6016d4b2cd None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:45.981413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:45.982683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:45.983176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:45.983957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:45.997954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.000323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.003039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.003817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.004167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.004508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.005346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.006764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:46.021486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:46.022433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:46.023410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:46.023410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:46.023410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:46.023410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:46.029375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 462/935] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:46.031301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 474/936] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:46.033079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.141509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dd5f1-61c2-40fe-9dcc-8819a91bf97a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.183176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.183885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.184161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.184432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.185185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:46.185489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.186861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.200545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 463/937] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:46.206670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.314653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec34835-8d5b-4299-a5ad-3d02121e5293 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.347681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.348845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.349321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.349785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.357517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.359160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-7c6375f9-a218-42a9-9cd5-2e6050a5a829 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:46.374245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:46.375202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:46.376133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.378653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 475/938] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:46.384408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.409978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b552f-58f1-447e-8327-aa2ebdd4a126 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.410252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b552f-58f1-447e-8327-aa2ebdd4a126 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.410457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b552f-58f1-447e-8327-aa2ebdd4a126 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.412092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 476/939] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:46.417116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.417259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.417492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.435520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:46.437030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:46.492148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc45df5-8c30-4867-9aeb-78ab10c07e4c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.508096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 477/940] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:46.531298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.533873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.535156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.535693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.536241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.537316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:46.537894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.539620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.559003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 464/941] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:46.567857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.646732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ea02d-d84e-4222-b9f5-9817ef67d772 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.664710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf928e5-6c72-4d5a-b8c2-8f9873321ed5 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.683660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.684851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.685132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.686959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.686959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.688269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.690161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING keystone.server.flask.common [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 16 18:38:46.699778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.700615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.700922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.701195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.711973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.712238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.sql.core [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 16 18:38:46.712238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 16 18:38:46.712238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [parameters: {'id': '0b5a7fe8b17c423b9bed03e43b8a188a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 16 18:38:46.712238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91543) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 16 18:38:46.713618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b5a7fe8b17c423b9bed03e43b8a188a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b5a7fe8b17c423b9bed03e43b8a188a. May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result = self._query(query) May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application conn.query(q) May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 18:38:46.731285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result.read() May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 16 18:38:46.732121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return project_ref.to_dict() May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application next(self.gen) May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application yield resource May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application next(self.gen) May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.session.rollback() May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.force_reraise() May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise self.value May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application session.commit() May 16 18:38:46.733074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._prepare_impl() May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.session.flush() May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._flush(objects) May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application compat.raise_( May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application flush_context.execute() May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 16 18:38:46.733948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rec.execute(self) May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application _emit_insert_statements( May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application c = connection._execute_20( May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ret = self._execute_context( May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 16 18:38:46.734833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result = self._query(query) May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application conn.query(q) May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result.read() May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 18:38:46.735603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [parameters: {'id': '0b5a7fe8b17c423b9bed03e43b8a188a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.736450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:46.739173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.Conflict( May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b5a7fe8b17c423b9bed03e43b8a188a. May 16 18:38:46.740077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:46.740681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:46.741465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:46.742343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:46.742343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:46.742343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:46.742343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:46.742559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 478/942] 10.222.0.36 () {60 vars in 1140 bytes} [Mon May 16 18:38:46 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:46.742626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 465/943] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:46.742701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.743870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.848355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77635e52-69f3-4fc0-8607-1adcbf8b0574 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.865940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeafeb-f0d4-4064-a039-e5163144338b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:46.885462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.886634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.887118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.887586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.895075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.897021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.905383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:46.906042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:46.906321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:46.906594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:46.907304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:46.907607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:46.909349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:46.914033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:46.914990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:46.915879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:46.923507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 479/945] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:46.924657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:46.926048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 466/945] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:46.927415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:47.036415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3819-4284-4b11-b84f-08e48a93080a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:47.049527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2728c82-5fa8-4ffb-ab59-46f71510a27f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:47.078959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:47.080001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.080359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.080730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.081915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:47.082306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:47.083797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:47.085713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:47.086702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.087159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.087626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.097077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:47.098805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:47.112669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 480/946] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:47.115357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:47.116263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:47.117189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:47.122718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:47.132767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 467/947] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:47.232403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64aea9-c905-4560-bedc-f8fc71fa4002 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:47.269084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:47.269834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.270120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.270395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.271208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:47.271514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:47.273021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e0b91-4562-4b7c-9224-edd412ae1f1d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:47.292220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 481/948] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:47.294185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:47.397247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224cd6-38f0-4600-828e-33c1571251cb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:47.431442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfa27d-8db1-4ef5-8892-3cf24a54612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:47.432321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfa27d-8db1-4ef5-8892-3cf24a54612f None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.432599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfa27d-8db1-4ef5-8892-3cf24a54612f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.432899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfa27d-8db1-4ef5-8892-3cf24a54612f None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/7bab459b1a3e48b0973482e9193aeb0e/roles/d9dde88ec4e44b569a261194c866b6e7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.435291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfa27d-8db1-4ef5-8892-3cf24a54612f None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=7bab459b1a3e48b0973482e9193aeb0e, role_id=d9dde88ec4e44b569a261194c866b6e7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:47.436752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfa27d-8db1-4ef5-8892-3cf24a54612f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:47.460368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 468/949] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:38:47 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/7bab459b1a3e48b0973482e9193aeb0e/roles/d9dde88ec4e44b569a261194c866b6e7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:38:47.694863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74efb-6ccb-4007-a20d-ec7ade580437 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.695413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74efb-6ccb-4007-a20d-ec7ade580437 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.695413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d74efb-6ccb-4007-a20d-ec7ade580437 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.697018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 482/950] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:47.703241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.703691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.703766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.726488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:47.727642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:47.793525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 469/951] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:47.815597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74efb-6ccb-4007-a20d-ec7ade580437 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:47.921610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d74efb-6ccb-4007-a20d-ec7ade580437 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:47.964059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:47.964545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:47.964896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:47.965167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:47.965814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:47.967179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:47.979362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:47.980455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:47.981343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 483/952] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:47.988717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:48.087219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2d7e2-5b74-4c4e-8a78-36d8816f87a2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:48.120845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:48.122021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.122293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.122559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.123257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:48.123555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:48.124978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:48.129931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 470/953] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:48.135365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:48.243344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76f718-28d9-44b3-9fc0-9fa9f13c381d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:48.279159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:48.279960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.280302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.280653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.281461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:48.281826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:48.283554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251b318-9318-4dcc-a65c-a00d312068e1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:48.288900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 484/954] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:48.294888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:48.391236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c1ff-f457-414d-8a8a-624d3a419439 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.391795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c1ff-f457-414d-8a8a-624d3a419439 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.392213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c1ff-f457-414d-8a8a-624d3a419439 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.394023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 485/955] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:48.397796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731cbb-fe5a-46d7-bed3-e84ad2ccd3ed None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:48.399518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.399983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.400403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.418910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:48.420570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:48.439107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:48.439979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.440398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.440839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.441708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:48.443201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:48.469015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 471/956] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:48.505082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 486/957] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:48.527884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:48.638266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a0609-07df-43f0-8185-6352f60ec9c2 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:48.675182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:48.676243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.676687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.677112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] PATH_INFO: `/v3/services/compute` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.677883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:48.679366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:48.691458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:48.692649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:48.693645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:48.695927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 472/958] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:48.702038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:48.805812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d873ea-08a8-4bfa-aabd-28daae574bbf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:48.841748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:48.842411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:48.842690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:48.842961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:48.843661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] RBAC: Adding query filter params (name=compute) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:48.843964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:48.845357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5712-a20b-4781-be3f-154431806caa None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:48.850578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 487/959] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:48.856027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:48.973127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77603353-dc73-49a7-8b77-8b88ed490edf None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:49.009960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:49.010622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:49.010897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:49.011169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:49.011903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:49.012261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:49.013877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bd6d3-7d19-4171-ba3a-62e38a2e3dd3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:49.019407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 473/960] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:49.553338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b58c5b-bcc7-4e37-859e-2639ee323b2f None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:49.553740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b58c5b-bcc7-4e37-859e-2639ee323b2f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:49.553879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b58c5b-bcc7-4e37-859e-2639ee323b2f None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:49.555510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 488/961] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:49.561071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97f60e-daca-48d0-a1fc-8c3f8278c3c5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:49.561363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97f60e-daca-48d0-a1fc-8c3f8278c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:49.561642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97f60e-daca-48d0-a1fc-8c3f8278c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:49.583328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6e97f60e-daca-48d0-a1fc-8c3f8278c3c5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:49.584540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6e97f60e-daca-48d0-a1fc-8c3f8278c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:49.656035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 474/962] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:49.681052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b58c5b-bcc7-4e37-859e-2639ee323b2f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:49.780664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50588695-c832-43ef-8543-dd4883229bdf None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:49.781005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50588695-c832-43ef-8543-dd4883229bdf None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:49.781234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50588695-c832-43ef-8543-dd4883229bdf None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:49.783071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 475/963] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:49.788145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:49.788434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:49.788539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b58c5b-bcc7-4e37-859e-2639ee323b2f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:49.788968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:49.805138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:49.806323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:49.828947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:49.829976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:49.829976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:49.830249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:49.831159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:49.833080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:49.850831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.sql.core [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 16 18:38:49.850831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 16 18:38:49.850831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [parameters: {'user_id': 'aa2813a7ddfa4f659ced522dbeae747a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 16 18:38:49.850831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91543) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result = self._query(query) May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application conn.query(q) May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result.read() May 16 18:38:49.860517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 18:38:49.861686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application next(self.gen) May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application yield resource May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application next(self.gen) May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.session.rollback() May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.force_reraise() May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise self.value May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application session.commit() May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 16 18:38:49.862774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._prepare_impl() May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.session.flush() May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._flush(objects) May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application compat.raise_( May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application flush_context.execute() May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rec.execute(self) May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 16 18:38:49.863728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application _emit_insert_statements( May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result = connection._execute_20( May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ret = self._execute_context( May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 16 18:38:49.864811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self.dialect.do_execute( May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result = self._query(query) May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application conn.query(q) May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application result.read() May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 16 18:38:49.865670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application packet.raise_for_error() May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [parameters: {'user_id': 'aa2813a7ddfa4f659ced522dbeae747a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:49.866582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 16 18:38:49.867677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:49.869178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 489/964] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 16 18:38:49 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:49.872060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:49.892780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 476/965] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:49.911275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:49.985150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97f2e-12f1-4bd7-a458-cb248d3bb595 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.018744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.019858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.020343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.020933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.022161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2352ada-d0e2-4b68-8980-0ff7c558597f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.029579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.031449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:50.046885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:50.047828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 16 18:38:50.049260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:50.051610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 490/966] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:38:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:50.057308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.059686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.060481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.060804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.061086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.073007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:38:50.084061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:50.085003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:38:50.085855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:50.088713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 477/967] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 16 18:38:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:50.093813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.191721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619ddff-e1eb-453a-b043-76f0dac0a383 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.196799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065f68f-ceb5-4203-bc32-022af567ee12 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.229801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.230386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.231047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.231325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.231530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.231629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.232081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.232309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:50.232617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.233104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.233991 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.234312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:50.234806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.236439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.247298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 478/968] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:50.248854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.281126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 491/969] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:50.354802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dbd78-51b4-4f43-b252-96d6920eb46c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.391446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.392241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.392512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.392809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.404421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.405915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:50.424717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:50.425726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:50.426980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:50.426980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:50.426980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:50.426980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:50.426980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 479/970] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:50.433997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.540875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b1ed-f530-4193-b4b6-cf87f12440cf None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.575354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.576012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.576292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.576563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.577305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:50.577606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.578953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.594812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 492/971] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:50.596146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.696238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509b42-e128-4126-abf9-717ee3f7adbe None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.727587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.728401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.728704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.728979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.737226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.738659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-005d47ce-b456-4c68-8b3f-85fe7ccb5419 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:38:50.753251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:50.754212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:50.755169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 480/972] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:50.760197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.824079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86278209-cd82-4591-aaea-a1df51630639 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.824384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86278209-cd82-4591-aaea-a1df51630639 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.824617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86278209-cd82-4591-aaea-a1df51630639 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.826128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 481/973] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:50.830964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.831254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.831525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.847951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:50.849141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:50.860355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61c1a-3d56-416f-a3ca-1aa5b0ee8323 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:50.894913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:50.895828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:50.896294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:50.896823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:50.898197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:50.898507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:50.899939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:50.918343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 482/974] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:50.919357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 493/975] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:50.924916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:50.942326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.022689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1614f1d-cb76-4c25-a768-4a7f371c6354 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.041876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ff4d5-72d1-460d-9402-31ef00980423 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.058108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.058944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.059365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.059773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.074973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.075766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.076043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.076309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.076991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.078345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:51.090902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:51.092229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:51.093267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 494/976] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:51.098429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.132683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.134428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.197097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e2e77-9eaf-4979-bce9-10891026edf4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.230236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.230829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.231084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.231372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.232045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:51.232424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.233734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.237889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 483/977] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:38:50 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:38:51.243046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 495/978] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:51.243173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.243786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.335071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db02b97-df3a-4ed1-8b8b-fdff3fe37acc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.337463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d2888-1c9d-4f99-bf34-62ad98e361e4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.366545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.367313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.367591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.367860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.368838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.369488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.369766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.370037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.370730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:51.371057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.372406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbaa40-0abb-45bb-adfe-b23bf78b5096 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.377050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 496/979] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:51.377279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.378389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060d4d9-0b95-4d66-8a34-d1cc2228e63d None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.378676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.378776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060d4d9-0b95-4d66-8a34-d1cc2228e63d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.378945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060d4d9-0b95-4d66-8a34-d1cc2228e63d None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.380659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 497/980] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:51.382571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060d4d9-0b95-4d66-8a34-d1cc2228e63d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-3c3319c6-9929-4a0a-8569-360a27f31015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:38:51.398148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:51.399198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:51.400271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:38:51.400271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:38:51.400271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:38:51.400271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:51.400271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 484/981] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:51.400608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.400897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.401172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.417484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:51.418682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:51.479977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 485/982] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:51.480893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.491557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060d4d9-0b95-4d66-8a34-d1cc2228e63d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.523827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.524471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.524782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.525056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.525831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] RBAC: Adding query filter params (interface=admin, service_id=80459b42c89c4a7c859ff21eb5c8bd6b, region_id=RegionOne) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:51.526136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.527831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.533663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 498/983] 10.222.0.36 () {56 vars in 1249 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/endpoints?service_id=80459b42c89c4a7c859ff21eb5c8bd6b&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:51.534417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.573443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e9b0e-c982-4645-a56c-46daeb1475c5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.604713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.605384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.605717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.605986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.606684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:51.606981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.608316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.618788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 486/984] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:51.619536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.628143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef306a-6294-4b93-b9d6-896a89836cc5 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.659940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.660583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.660902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.661173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.661871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.663210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.676585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 499/985] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:51 2022] POST /identity/v3/services => generated 242 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:51.677621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.721372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee53cbc-d8ea-4fb6-8088-194bb8745a94 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.768921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.769499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.769775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.770043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.770731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.772107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.778887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 487/986] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:38:51.784046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5388-14f0-4aca-b382-84590e44d525 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:51.822361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:51.823192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:51.823475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:51.823750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:51.832085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:51.833648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:38:51.849307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:51.850303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:51.851435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 500/987] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:51.861197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:51.980970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0d0da-6d99-4cf3-ab2e-561a9fb1727c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:52.021779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:52.022791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:52.023271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:52.023739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:52.024785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:52.025292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:52.026913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56893b9-37b1-42f7-8c1f-3d8a6d7ffb18 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:52.040431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 488/988] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:52.046478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:52.147771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ee524-067c-48d5-be1d-938b0e982754 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:52.183966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:52.185159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:52.185437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:52.185703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:52.186528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] RBAC: Adding query filter params (role.id=d9dde88ec4e44b569a261194c866b6e7, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=7bab459b1a3e48b0973482e9193aeb0e) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:52.186841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:52.188267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0af86a-be6b-40ad-9df7-68bb3c426d2b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:52.194601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 501/989] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:38:52 2022] GET /identity/v3/role_assignments?user.id=7bab459b1a3e48b0973482e9193aeb0e&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=d9dde88ec4e44b569a261194c866b6e7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:53.981616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cd054-4b34-4f61-ac10-0d9581eb3494 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:53.982140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cd054-4b34-4f61-ac10-0d9581eb3494 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:53.982140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cd054-4b34-4f61-ac10-0d9581eb3494 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:53.983549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 489/990] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:53.988747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544294-e843-449b-8ba8-97cf70af1fab None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:53.989152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544294-e843-449b-8ba8-97cf70af1fab None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:53.989313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544294-e843-449b-8ba8-97cf70af1fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.005745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ee544294-e843-449b-8ba8-97cf70af1fab None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:54.007252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ee544294-e843-449b-8ba8-97cf70af1fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:54.072768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 502/991] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:54.085640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cc540-9d8c-478d-80f9-758fbcd94bac None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.085930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cc540-9d8c-478d-80f9-758fbcd94bac None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.086206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cc540-9d8c-478d-80f9-758fbcd94bac None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.087667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 490/992] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:54.093330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.093622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.093901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.095464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cc540-9d8c-478d-80f9-758fbcd94bac None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.110897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:54.112062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:54.177596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 503/993] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:54.198416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cc540-9d8c-478d-80f9-758fbcd94bac None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.200787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.232826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.233534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.233826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.234087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.234884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.236243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:54.247779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:54.249006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:54.250447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 491/994] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:54.254915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.315827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2015b-12d7-406b-8ff0-e84560a85b88 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.349635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.350442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.350721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.350987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.351649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.353014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.362818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bc7f7-d5db-4b6c-b9b6-e4454e20aa75 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:54.364670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:54.365638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:54.366472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 504/995] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:54.371799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.400234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.400918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.401198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.401476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.402720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:54.403024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.404381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.414015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 492/996] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:54.415055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.469877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d9bd-d192-4ec4-a187-a0f9bad45bbe None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.501706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.502298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.502577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.502855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.503555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:54.503868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.505268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7b760-f0ea-4790-9745-4bd5ccf5ca30 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.509862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 505/997] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:54.509992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a914d0-66e8-426c-bef5-dec4e3a569cc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.511039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf714a4-377d-4cdc-a39a-a401b6299d02 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.511395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf714a4-377d-4cdc-a39a-a401b6299d02 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.512162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf714a4-377d-4cdc-a39a-a401b6299d02 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.513409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 506/998] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:54.516158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf714a4-377d-4cdc-a39a-a401b6299d02 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.542870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.543525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.543830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.544135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.544890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:54.545199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.546559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af7479-5d46-4922-93ef-4989cb832bd5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.551324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 493/999] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:54.552317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.552603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.552912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.569174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:54.570348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:54.617723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf714a4-377d-4cdc-a39a-a401b6299d02 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.632498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 494/1000] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:54.633352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.662415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.663071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.663355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.663631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.664332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:54.664649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.666030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.670854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 507/1001] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:54.671447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.726933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f35c6-9941-430d-8f90-4015730c626b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.758241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.758814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.759089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.759356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.760121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] RBAC: Adding query filter params (interface=public, service_id=eaee4ade957445cca40975e3b285bfbd, region_id=RegionOne) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:54.760423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.762348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.765725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f275590-ec1d-440c-aa9f-d03c127c6386 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.771629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 495/1002] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/endpoints?service_id=eaee4ade957445cca40975e3b285bfbd&interface=public®ion_id=RegionOne => generated 194 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:54.771629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.797593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.798367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.798646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.798920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] PATH_INFO: `/v3/services/image` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.799559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.800945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:54.812652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:54.813832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:38:54.814713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 508/1003] 10.222.0.36 () {56 vars in 1105 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services/image => generated 86 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:38:54.815530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.864173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f154-47f4-4825-a180-c2049c1201bc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.896429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.897091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.897364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.897639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.898366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.899704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.909268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a1659-b4a1-4069-a476-6274c4ffdfe1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:54.915014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 496/1004] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:54.915866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:54.954594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:54.958234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:54.958234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:54.958234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:54.958234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:54.958234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:54.966270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 509/1005] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:38:55.055875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497be151-78d9-4918-a4c7-47a7ae01e40f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:55.093291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:55.093980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:55.094260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:55.094532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:55.095253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] RBAC: Adding query filter params (name=image) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:55.095557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:55.096986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10ae96-8494-4d28-b833-de0cedb7fc1e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:55.107758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 497/1006] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:55.108979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:55.214175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e66d6-954a-4b3a-a5a3-10d4c74e96e7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:55.246882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:55.247535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:55.247813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:55.248079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:55.248796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] RBAC: Adding query filter params (type=image) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:55.249100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:55.250450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0ca7-fea6-4dc8-a75a-522f7a66a62a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:55.254974 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 510/1007] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:57.190744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc486c-3614-4105-bb35-563ebf995a33 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.191287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc486c-3614-4105-bb35-563ebf995a33 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.191287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc486c-3614-4105-bb35-563ebf995a33 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.192881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 498/1008] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:57.197750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.198141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.198313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.215695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:57.217152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:57.217537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10fb52-5f82-4b88-a870-0f6d05297186 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.217827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10fb52-5f82-4b88-a870-0f6d05297186 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.218099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10fb52-5f82-4b88-a870-0f6d05297186 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.219441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 499/1009] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:38:57.225824 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.226114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.226760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.245796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:57.247232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:57.281758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 511/1010] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:57.304884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.318242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 500/1011] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:38:57.341972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.403947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f93e77-f412-4e91-82b3-d936793ca587 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.434225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea46d-2b19-43f8-8784-d4f175c89269 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.435997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.436830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.437111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.437380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.438567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.439988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.441638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 512/1012] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:38:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:57.447282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.469166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.469874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.470151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.470417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.471050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.472393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:38:57.483589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:38:57.484556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:38:57.490301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 501/1013] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:38:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:38:57.491169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.548066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c480f9e-903e-4be9-8f68-d7b8365be4e7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.580559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60c998-4522-497d-b55f-0f9f73e45d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.581256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60c998-4522-497d-b55f-0f9f73e45d21 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.581534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60c998-4522-497d-b55f-0f9f73e45d21 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.581807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60c998-4522-497d-b55f-0f9f73e45d21 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.582501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60c998-4522-497d-b55f-0f9f73e45d21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.582679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081a613-481c-4ef5-81bd-4f47eb522dc1 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.583916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60c998-4522-497d-b55f-0f9f73e45d21 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.608351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 513/1014] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:57 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:38:57.609635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c5d29-27b4-4dbe-899a-f5b2bd533a73 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.609917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c5d29-27b4-4dbe-899a-f5b2bd533a73 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.610184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c5d29-27b4-4dbe-899a-f5b2bd533a73 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.611540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 514/1015] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:57.617237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.617529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.617787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.619738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.620421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.621289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.621555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.622308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] RBAC: Adding query filter params (name=compute) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:57.622620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.624048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.634203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 502/1016] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:38:57.635232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.636175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:57.638135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:57.724566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 515/1017] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:38:57.748125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.749983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5a2b2-92ba-44c0-989e-59bbcce8988d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.788598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.789271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.789551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.790227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.790514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] RBAC: Adding query filter params (type=compute) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:38:57.790838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.792197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.797801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 503/1018] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 16 18:38:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:38:57.802996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:38:57.854656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1bf81-735e-4446-9e57-f72ecaa435e5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.891461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8db6e-4835-4f2d-9c08-7b337257465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.892129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8db6e-4835-4f2d-9c08-7b337257465a None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.892407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8db6e-4835-4f2d-9c08-7b337257465a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.892705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8db6e-4835-4f2d-9c08-7b337257465a None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.893427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8db6e-4835-4f2d-9c08-7b337257465a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.894781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8db6e-4835-4f2d-9c08-7b337257465a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.906274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2db1a-d850-448c-8a68-97c51ec32f6b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:38:57.915127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 516/1019] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:38:57 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:38:57.948106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5157a-7040-4b5a-bfa9-e960eebe66d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:38:57.948881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5157a-7040-4b5a-bfa9-e960eebe66d5 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:57.949164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5157a-7040-4b5a-bfa9-e960eebe66d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:57.949434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5157a-7040-4b5a-bfa9-e960eebe66d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:57.950170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5157a-7040-4b5a-bfa9-e960eebe66d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:38:57.951583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5157a-7040-4b5a-bfa9-e960eebe66d5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:38:57.980745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 504/1020] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:38:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:38:59.883451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727c9ad-d3c6-4a7a-bf0a-48442e82a8cd None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:59.884008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727c9ad-d3c6-4a7a-bf0a-48442e82a8cd None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:59.884008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727c9ad-d3c6-4a7a-bf0a-48442e82a8cd None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:59.885523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 517/1021] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:38:59.890808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674664f-539a-4a9f-9ba5-1edf3b165940 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:38:59.891201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674664f-539a-4a9f-9ba5-1edf3b165940 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:38:59.891378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674664f-539a-4a9f-9ba5-1edf3b165940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:38:59.909182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7674664f-539a-4a9f-9ba5-1edf3b165940 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:38:59.910362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7674664f-539a-4a9f-9ba5-1edf3b165940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:38:59.982852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 505/1022] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:38:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:00.007584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727c9ad-d3c6-4a7a-bf0a-48442e82a8cd None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.144680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727c9ad-d3c6-4a7a-bf0a-48442e82a8cd None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.189523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88279a-c7d4-41f4-92e6-18f4e107624e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.190006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88279a-c7d4-41f4-92e6-18f4e107624e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.190487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88279a-c7d4-41f4-92e6-18f4e107624e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.196462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 506/1023] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:00.196602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.196602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.196602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.196602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.197207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.199039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.199139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.199391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.199728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.200914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 518/1024] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:00.206024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.221791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:00.222967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:00.294383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 507/1025] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:00.310302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005af5-1f64-4c78-aef1-deed8ab03892 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.315334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.354848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d3739-dd32-4866-9d29-0723e3a97c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.355739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d3739-dd32-4866-9d29-0723e3a97c87 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.356019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d3739-dd32-4866-9d29-0723e3a97c87 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.356339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d3739-dd32-4866-9d29-0723e3a97c87 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.357033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d3739-dd32-4866-9d29-0723e3a97c87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.358482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d3739-dd32-4866-9d29-0723e3a97c87 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.388676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 519/1026] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:39:00 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:39:00.425940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe87f7-ac46-47f2-83ef-d4189c836777 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.438592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dddea-2e6a-4155-b286-3a1aa8edd851 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.438932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dddea-2e6a-4155-b286-3a1aa8edd851 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.439216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dddea-2e6a-4155-b286-3a1aa8edd851 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.440649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 520/1027] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:39:00.446574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.446859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.447138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.465644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:00.466911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:00.467455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.468340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.468605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.468911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] PATH_INFO: `/v3/services/image` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.469569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.470952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:00.482796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:00.483912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:00.490609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 508/1028] 10.222.0.36 () {56 vars in 1105 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:00.494234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.538809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 521/1029] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:00.562608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.600548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4b4f-7e88-48ea-bbce-15554674fdbd None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.635111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.635774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.636649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.636911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.637613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:00.637918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.639281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.644553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 509/1030] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:00.650280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.665255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f8652-cf5b-4769-aa74-e99746fa5bee None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.701630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.702362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.702644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.702917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.713620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:00.721736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:00.722672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:00.723497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 522/1031] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:00.728673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.751469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598c539-7b20-408f-a20c-925d790654d3 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.785065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.785714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.786110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.786392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.787088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] RBAC: Adding query filter params (type=image) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:00.787389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.788757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.796668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 510/1032] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:00.799044 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.827743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b04e8a-d8ae-47f6-87e2-e63365545112 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.860518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.861444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.861934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.862345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.863208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:00.863649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.865180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.873985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 523/1033] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:00.879158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.894060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec316dd-af7f-40c5-849f-062455893d1f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:00.930939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:00.931590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:00.931869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:00.932141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:00.933585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] RBAC: Adding query filter params (interface=public, service_id=ebff8efdd19044d1ad9f0d21cdb76a4b, region_id=RegionOne) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:00.933894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:00.935250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:00.944839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 511/1034] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/endpoints?service_id=ebff8efdd19044d1ad9f0d21cdb76a4b&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:00.945675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:00.979400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe3aae-2a05-43ee-983f-0b217246c44c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.012087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.012894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.013167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.013429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.023623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.025097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:01.041673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:01.043324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:01.044512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:01.044512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:01.044512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:01.044512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:01.044512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 524/1035] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:01.047946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407c633-eb09-4000-98b1-aee9f22b26ba None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.049323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:01.081702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.082413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.082692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.083040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.083706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.085127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.091368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 512/1036] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:39:00 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:39:01.148695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09f3bc-2fe3-47ba-baf5-2130ead7a4a1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.184666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.185339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.185616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.185889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.186618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:01.186920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.188323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.205802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 525/1037] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:01.207050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:01.310110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c0934-0325-4e97-8f1d-a416755d7aee None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.342355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.343132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.343412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.343685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.350925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.352340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:39:01.366334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:01.367381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:01.368359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 513/1038] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:39:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:01.375312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:01.470172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca2174-c71e-4b00-b72d-09cfc3e81f55 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.502758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.503420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.503701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.503974 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.504829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:01.505138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.506556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.524258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 526/1039] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:39:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:01.530222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:01.627651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf792bf1-da52-491b-ab14-a3a2cee60fd0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.660226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.660900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.661185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.661459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.725665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.727110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.820385 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 514/1040] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:01.826399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:01.923817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622ffc-3954-47b0-8c00-c65bbe181dba None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:01.960599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:01.960599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:01.960599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:01.960599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:01.971584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:01.973049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:01.990416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:01.996708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:01.997636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:01.997636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:01.997636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:01.997636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:01.997636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 527/1041] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:01.999418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:02.093364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbb932-b5a6-4c1a-bec8-8c9ce37dcfe0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:02.125148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:02.125806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.126089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.126363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.127074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:02.127388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:02.128757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:02.144419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 515/1042] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:02.146159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:02.241807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db0348-338c-4bc2-aaf7-9c61b27e2b59 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:02.274578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:02.275356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.275633 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.275891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.282350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:02.284042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:39:02.297228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:02.298159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:39:02.299060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:02.303144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 528/1043] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:02.304556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:02.397855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bc797-7bd6-45cd-84ea-9214c5ec238f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:02.429210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:02.429849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.430126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.430396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.431159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:02.431460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:02.433414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af71dc-fac0-493c-9784-bc2c6ec7355c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:02.444676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 516/1044] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:02.450620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:02.548753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812821bb-22e9-45d2-a177-26653d449cd0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:02.580286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:02.580961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.581240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.581510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.582316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] RBAC: Adding query filter params (role.id=d9dde88ec4e44b569a261194c866b6e7, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=b163f5034b7a4d96a224c07b9c1428c3) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:02.582619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:02.584028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa39c92-4a66-4f28-968f-9791577395cd None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:02.596668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 529/1045] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:02 2022] GET /identity/v3/role_assignments?user.id=b163f5034b7a4d96a224c07b9c1428c3&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=d9dde88ec4e44b569a261194c866b6e7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:02.681049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919018eb-0ad3-49d8-ac72-39364839f183 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.681331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919018eb-0ad3-49d8-ac72-39364839f183 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.681605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919018eb-0ad3-49d8-ac72-39364839f183 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.683552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 517/1046] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:02.689364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.689650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.689932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.707306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:02.708521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:02.780149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 530/1047] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:02.804307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919018eb-0ad3-49d8-ac72-39364839f183 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:02.900544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919018eb-0ad3-49d8-ac72-39364839f183 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:02.935868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:02.936670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:02.936955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:02.937226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:02.947024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:02.956748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:02.957704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:02.958534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:02.965081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 518/1048] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:02.965884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.061765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3b5eb-18cc-47b5-ac00-d525c6041a94 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.094423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.095089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.095371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.095652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.096382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:03.096715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.098076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.110958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 531/1049] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:03.112185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.211895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df88517-5289-4704-aaef-d7bd5c776608 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.243718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e5f17-ffb1-4500-acfd-902fc6bf071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.244583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e5f17-ffb1-4500-acfd-902fc6bf071a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.244895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e5f17-ffb1-4500-acfd-902fc6bf071a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.245170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e5f17-ffb1-4500-acfd-902fc6bf071a None admin] PATH_INFO: `/v3/users/cb906e3a8c1043cd8248ff856a7ee721` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.257936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e5f17-ffb1-4500-acfd-902fc6bf071a None admin] RBAC: Authorizing `identity:get_user(user_id=cb906e3a8c1043cd8248ff856a7ee721)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.259371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e5f17-ffb1-4500-acfd-902fc6bf071a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.261748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 519/1050] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/users/cb906e3a8c1043cd8248ff856a7ee721 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:03.267245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.312822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ced68-1629-4c78-9b21-cc89993dd97c None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.313118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ced68-1629-4c78-9b21-cc89993dd97c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.313301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ced68-1629-4c78-9b21-cc89993dd97c None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.314641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 520/1051] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:39:03.319528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56943130-2088-404a-8cc9-554aed595821 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.319813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56943130-2088-404a-8cc9-554aed595821 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.320086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56943130-2088-404a-8cc9-554aed595821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.337169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-56943130-2088-404a-8cc9-554aed595821 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:03.338328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-56943130-2088-404a-8cc9-554aed595821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:03.364979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040d90d-d619-40cb-9a21-3a9e9ef2a2b7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.397856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.398509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.398787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.399064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.403818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 521/1052] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:03.425853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56943130-2088-404a-8cc9-554aed595821 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.465720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.467155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.520406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56943130-2088-404a-8cc9-554aed595821 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.553501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.554273 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.554562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.554838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] PATH_INFO: `/v3/services/image` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.555504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.556879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.561721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 532/1053] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:03.567277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:03.568759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:03.569925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:03.570766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 522/1054] 10.222.0.36 () {56 vars in 1105 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:03.576090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.669971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf9fab-781b-4a8c-b9f1-e63b9a6abf28 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.671162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6e38-7737-4f98-b849-aa3bf5227ceb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.705643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.706542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.706725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.706841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.707077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.707326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.707615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.707880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.708234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] RBAC: Authorizing `identity:get_project(project_id=dda24e6463a545ed985165c26beb467c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.708584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] RBAC: Adding query filter params (name=image) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:03.708923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.709689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.710261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.719648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 523/1055] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:03.719811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 533/1056] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/projects/dda24e6463a545ed985165c26beb467c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:03.719914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.720374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.814486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8919d6e-8f08-4114-9e5c-498b13daea24 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.819747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061a4b-7f82-4dd3-8fec-f7810cf6151a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:03.850668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.851697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.852286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.852812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.853839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=dda24e6463a545ed985165c26beb467c, user.id=cb906e3a8c1043cd8248ff856a7ee721) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:03.854373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.854895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:03.855552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:03.855835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:03.856110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:03.856847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:03.857156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:03.857392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af808aa-bd0b-48f9-9fca-6b2b70545bb2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.858504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:03.863353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 534/1057] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:03.863698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 524/1058] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:03 2022] GET /identity/v3/role_assignments?user.id=cb906e3a8c1043cd8248ff856a7ee721&scope.project.id=dda24e6463a545ed985165c26beb467c&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:03.869019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:03.980365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a49e34-bd1e-41c0-b19f-4aa6014615d8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:04.015263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f67a-b1f8-457a-85af-bcb92a2bec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:04.015930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f67a-b1f8-457a-85af-bcb92a2bec5b None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:04.016212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f67a-b1f8-457a-85af-bcb92a2bec5b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:04.016487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f67a-b1f8-457a-85af-bcb92a2bec5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:04.017280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f67a-b1f8-457a-85af-bcb92a2bec5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:04.018675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699f67a-b1f8-457a-85af-bcb92a2bec5b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:04.038105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 535/1059] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:39:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:39:04.792330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366697-c71a-4978-b0a1-2a1a070a9c26 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:04.792889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366697-c71a-4978-b0a1-2a1a070a9c26 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:04.792889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366697-c71a-4978-b0a1-2a1a070a9c26 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:04.794532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 525/1060] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:04.799575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:04.800027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:04.800107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:04.816325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:04.817525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:04.886561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 536/1061] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:04.910249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366697-c71a-4978-b0a1-2a1a070a9c26 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.010409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366697-c71a-4978-b0a1-2a1a070a9c26 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.043098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.043964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.047662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.047662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.059629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:05.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:05.069078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:39:05.069979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:05.073057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 526/1062] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 16 18:39:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:05.075653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.169321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c13d4-393a-44dc-a8c7-0e617b39f755 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.203984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.204668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.204950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.205220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.205960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:05.206264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:05.207628 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:05.215526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 537/1063] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:39:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:05.221189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.315468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba19ab-02d9-4751-a8c8-96a4e6477756 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.348033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.348834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.349112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.349379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.359197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:05.360624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:05.378022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:05.378952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:05.379872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:05.379872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:05.379872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:05.379872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:05.381029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 527/1064] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:05.384750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.480981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31595b2-8911-4a48-bb47-df1f7884ff1b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.512898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.513551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.513829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.514100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.514810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:05.515110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:05.516464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:05.532050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 538/1065] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:05.534124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.630308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c9a70-7cfd-46be-b8ad-39fb9c8e37df None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.662590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.663650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.664000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.664445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.672104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:05.673632 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:39:05.689325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:05.690335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:39:05.691295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:05.692954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 528/1066] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:39:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:05.696402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.791103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918554c-0aec-4ca4-a24b-894a617785d4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.823203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.823959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.824155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.824425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.825270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:05.825574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:05.826991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:05.848212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 539/1067] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:39:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:05.849984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:05.951999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d63163-6fd2-4d02-a7c1-ed874a6d22ad None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:05.985741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:05.986536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:05.986822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:05.987099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:05.996674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:05.998340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-4da804ff-4473-43c0-be8f-6c7ea614dfc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:06.015199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:06.016117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:06.017056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:06.017056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:06.017056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:06.017056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:06.020951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 529/1068] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:06.022221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.062058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100356-06bd-4993-b75d-1895fade45e8 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.062378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100356-06bd-4993-b75d-1895fade45e8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.062616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100356-06bd-4993-b75d-1895fade45e8 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.064016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 530/1069] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:06.068975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.069261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.069536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.085592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:06.086745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:06.128793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348167b6-0431-4eba-8309-adbeecb165a8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.156690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 531/1070] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:06.164698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.165323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.165603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.165875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.166620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:06.166924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.168360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.178556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.183751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 540/1071] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:06.185378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.273696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d50bb-fcf7-48cf-9d9c-7113716e08b2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.282356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594541ae-dd27-4ebc-93f7-babd685e0dbd None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.306608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d086f6-7809-4036-84e6-473d3d31ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.307507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d086f6-7809-4036-84e6-473d3d31ec38 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.307768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d086f6-7809-4036-84e6-473d3d31ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.308037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d086f6-7809-4036-84e6-473d3d31ec38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.315600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.316382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.316681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.316956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.317848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d086f6-7809-4036-84e6-473d3d31ec38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:06.323858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.325325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-08d086f6-7809-4036-84e6-473d3d31ec38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:06.325959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:06.327029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:06.327868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 532/1072] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:06.328554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789060d-3e3f-4b7c-ac8f-b9de93da76ec None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.328963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789060d-3e3f-4b7c-ac8f-b9de93da76ec None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.329058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789060d-3e3f-4b7c-ac8f-b9de93da76ec None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.330416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 533/1073] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:06.333002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789060d-3e3f-4b7c-ac8f-b9de93da76ec None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-c797d39a-d9db-4e51-aed2-16b09bdd603f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:39:06.339796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:06.340925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 541/1074] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.342444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.343865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.358496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:06.359686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:06.423068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 542/1075] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:06.424099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.430480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789060d-3e3f-4b7c-ac8f-b9de93da76ec None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.465120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.465769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.466045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.466315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.467045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:06.467342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.468704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.476903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 534/1076] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:06.477530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.518108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1fce9-f97f-430d-9108-ab39972d094f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.549649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.550300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.550578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.550848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.551618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:06.551928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.553383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.565008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 543/1077] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:06.565867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.569142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec64f-27b6-4878-8c31-6ea0ae46859a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.601380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.602017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.602294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.602568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.661908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024f38-80e0-4e42-b0aa-0a1d975837fd None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.667370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.668843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.693221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.694067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.694345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.694612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] PATH_INFO: `/v3/users/cb906e3a8c1043cd8248ff856a7ee721` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.695783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] RBAC: Authorizing `identity:get_user(user_id=cb906e3a8c1043cd8248ff856a7ee721)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.697213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.699363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 544/1078] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/users/cb906e3a8c1043cd8248ff856a7ee721 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:06.700076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.759664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 535/1079] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:06.760484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.793717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c409e6-f591-4ce2-b48a-d387d26a82dc None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.825893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.826720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.826999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.827275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/b163f5034b7a4d96a224c07b9c1428c3/roles/d9dde88ec4e44b569a261194c866b6e7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.829516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=b163f5034b7a4d96a224c07b9c1428c3, role_id=d9dde88ec4e44b569a261194c866b6e7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.830940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.843432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 545/1080] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:39:06 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/b163f5034b7a4d96a224c07b9c1428c3/roles/d9dde88ec4e44b569a261194c866b6e7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:39:06.844222 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.853333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe286b-ef93-4380-aedd-7922240d679f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.891172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.892036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.892367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.893001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:06.893918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] RBAC: Authorizing `identity:get_project(project_id=dda24e6463a545ed985165c26beb467c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:06.895360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:06.902320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 536/1081] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/projects/dda24e6463a545ed985165c26beb467c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:06.903486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:06.952931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705161f0-13a5-42c4-ab02-8750873f04cb None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:06.991875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:06.992869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:06.993284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:06.993650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:07.005624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:07.007262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:07.012749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87b2a-c3bd-4f18-9fc9-c8336423f2e4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:07.027568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:07.028525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:07.029461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:07.029461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:07.029461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:07.029461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:07.029461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 546/1082] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:07.036441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:07.050435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bb6af-8737-4d4b-b36f-e238d81710c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:07.051323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bb6af-8737-4d4b-b36f-e238d81710c0 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:07.051602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bb6af-8737-4d4b-b36f-e238d81710c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:07.051876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bb6af-8737-4d4b-b36f-e238d81710c0 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c/users/cb906e3a8c1043cd8248ff856a7ee721/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:07.053846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bb6af-8737-4d4b-b36f-e238d81710c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda24e6463a545ed985165c26beb467c, user_id=cb906e3a8c1043cd8248ff856a7ee721, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:07.055263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bb6af-8737-4d4b-b36f-e238d81710c0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:07.076749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 537/1083] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:39:06 2022] PUT /identity/v3/projects/dda24e6463a545ed985165c26beb467c/users/cb906e3a8c1043cd8248ff856a7ee721/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:39:07.143870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d745d9-4c90-46d7-b6f4-e1d9c896534e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:07.179945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:07.180712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:07.180995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:07.181267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:07.181992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:07.182295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:07.184309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b62826-9436-405d-8303-43f9a10285d7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:07.198072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 547/1084] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:09.084163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643c5a7-16a2-4156-a252-315753c74a6b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.084718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643c5a7-16a2-4156-a252-315753c74a6b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.084884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643c5a7-16a2-4156-a252-315753c74a6b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.086406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 538/1085] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:09.092948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e4a9-a398-4c3b-938c-2e102c47f71a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.093390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e4a9-a398-4c3b-938c-2e102c47f71a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.093493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e4a9-a398-4c3b-938c-2e102c47f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.109899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-2b16e4a9-a398-4c3b-938c-2e102c47f71a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:09.111090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2b16e4a9-a398-4c3b-938c-2e102c47f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:09.187434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 548/1086] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:09.211169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643c5a7-16a2-4156-a252-315753c74a6b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.319329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643c5a7-16a2-4156-a252-315753c74a6b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.331914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7492ad-5613-4108-9f4b-687812435d49 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.332204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7492ad-5613-4108-9f4b-687812435d49 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.332479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7492ad-5613-4108-9f4b-687812435d49 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.333876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 549/1087] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:39:09.339473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.339752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.340020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.356229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:09.356682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.357433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:09.357580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.357861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.358131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.368809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:09.378010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:09.378954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:09.379785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 539/1088] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:09.386480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.430041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 550/1089] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:09.453452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.486566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c0b46-e7bd-4453-9082-abf99c9a7483 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.520310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.520990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.521270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.521538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.522268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:09.522566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:09.523906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:09.532036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 540/1090] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:09.537891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.559613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e4ed5-69fb-4855-ab06-3b8a37a22f25 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.598773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.599725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.600006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.600272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.611132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:09.619882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:09.621384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:09.622327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 551/1091] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:09.628116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.647473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad5652-75c1-4fcb-8afa-07ee9b4b659f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.680922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.681707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.681983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.682251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.693079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:09.694506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:09.711575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:09.712600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:09.713580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:09.713580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:09.713580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:09.713580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:09.713580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 541/1092] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:09.718554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.735538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91962bb8-f3b8-4ffe-9acd-906419fed429 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.771724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.772306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.772585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.772885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.773628 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:09.773933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:09.775950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:09.785515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 552/1093] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:09.791300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.815043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b8b0d-5054-4a60-b123-fa78a37d6c2a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.854206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.855558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.855839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.856107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.856859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:09.857163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:09.858522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:09.883116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 542/1094] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:09.889381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.897591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b58231-1579-4983-8f5a-97ac7d2090a1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:09.934572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:09.935438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:09.935716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:09.935986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] PATH_INFO: `/v3/users/cb906e3a8c1043cd8248ff856a7ee721` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:09.937212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] RBAC: Authorizing `identity:get_user(user_id=cb906e3a8c1043cd8248ff856a7ee721)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:09.938640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:09.940279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 553/1095] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/users/cb906e3a8c1043cd8248ff856a7ee721 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:09.945872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:09.996268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512020c2-3db9-4e6c-8ab4-54677c11b183 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.028325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.029147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.029432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.029708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.037125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.038575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.050782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb84573-b3a6-491f-bdb8-bb850a4f722c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:39:10.054800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:10.055790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:10.056765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 543/1096] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:39:10.062483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.085581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.086218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.086497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.086773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.150420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.151941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.154725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401eac0-ec05-4151-bef3-d44d267b9f62 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.187348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.188003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.188285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.188559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.189403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:10.189708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.191109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.207391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 544/1097] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:10.213061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.244832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 554/1098] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:39:10.251095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.305190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eea45-8786-4a4e-9829-f23e56cd70dc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.336217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.336864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.337148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.337422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.341029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396465da-fcc6-4b28-977d-19f528aec642 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.373158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.374000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.374275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.374543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.375693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] RBAC: Authorizing `identity:get_project(project_id=dda24e6463a545ed985165c26beb467c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.377149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.378827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 555/1099] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/projects/dda24e6463a545ed985165c26beb467c => generated 311 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:10.384921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.402027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.403443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.477132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edd485-3d37-4fe9-858c-5bde8bd85e16 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.499795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 545/1100] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:39:10.505561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.509161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.509796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.510073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.510343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.511155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=dda24e6463a545ed985165c26beb467c, user.id=cb906e3a8c1043cd8248ff856a7ee721) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:10.511459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.512900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.518620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 556/1101] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/role_assignments?user.id=cb906e3a8c1043cd8248ff856a7ee721&scope.project.id=dda24e6463a545ed985165c26beb467c&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:10.603581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03427-5e4e-41d3-89d0-36b908f5c60a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.638242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.639041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.639322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.639604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.650211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.652263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:10.671814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:10.672982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:10.674723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:39:10.674723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:10.674723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:39:10.674723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:10.674723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 546/1102] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:10.678990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.782093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29a4bd-0518-4498-b3ab-28f03cdd67f0 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.813725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.814379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.814660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.814933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.815646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:10.815952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.817342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.829838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 557/1103] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:10.835634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:10.932965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4029d-d983-4d75-9fe9-2efea20502a5 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:10.965597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:10.966380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:10.966656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:10.966924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:10.973263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:10.974697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:39:10.987774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:10.988744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:10.989644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 547/1104] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:10.994760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:11.093074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab69d2-765d-42d0-92b5-88067cbbd980 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:11.125553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:11.126202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:11.126481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:11.126755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:11.127558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:11.127863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:11.129327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b5fe7-8d8d-4a2c-87b1-4c5bf6f2954a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:11.140901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 558/1105] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:11.147021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:11.239405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4713c78-485b-4e58-ac45-a2ea5ffcf637 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:11.271170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:11.271810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:11.272085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:11.272360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:11.273200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] RBAC: Adding query filter params (role.id=d9dde88ec4e44b569a261194c866b6e7, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=b163f5034b7a4d96a224c07b9c1428c3) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:11.273508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:11.274912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8894ec5-8a5f-4505-bf18-5ddf982d0472 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:11.280479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 548/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:11 2022] GET /identity/v3/role_assignments?user.id=b163f5034b7a4d96a224c07b9c1428c3&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=d9dde88ec4e44b569a261194c866b6e7 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:12.776722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539901-b7b8-4197-b0e8-57816ee1cdbb None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:12.777280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539901-b7b8-4197-b0e8-57816ee1cdbb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:12.777280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539901-b7b8-4197-b0e8-57816ee1cdbb None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:12.778639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 559/1107] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:12.784133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:12.784560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:12.784691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:12.800648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:12.801817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:12.863421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 549/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:12.888764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539901-b7b8-4197-b0e8-57816ee1cdbb None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:12.986522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539901-b7b8-4197-b0e8-57816ee1cdbb None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:13.017560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d360d8-0a50-498a-95e5-dec9630f8495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:13.018324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d360d8-0a50-498a-95e5-dec9630f8495 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:13.018601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d360d8-0a50-498a-95e5-dec9630f8495 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:13.018869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d360d8-0a50-498a-95e5-dec9630f8495 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:13.020010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d360d8-0a50-498a-95e5-dec9630f8495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:13.021468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d360d8-0a50-498a-95e5-dec9630f8495 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:13.023059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 560/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:13.028924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:13.119334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64267f83-d9f5-4e1c-84dc-6b0119ba7c4b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:13.150426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f596ef-34f6-4bf4-9729-993782398626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:13.151070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f596ef-34f6-4bf4-9729-993782398626 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:13.151368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f596ef-34f6-4bf4-9729-993782398626 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:13.151634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f596ef-34f6-4bf4-9729-993782398626 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:13.152325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f596ef-34f6-4bf4-9729-993782398626 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:13.153759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f596ef-34f6-4bf4-9729-993782398626 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:13.172986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 550/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:39:13 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:39:15.411374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9f58b-7030-452a-9394-9657cb7a169d None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:15.411940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9f58b-7030-452a-9394-9657cb7a169d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:15.411940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9f58b-7030-452a-9394-9657cb7a169d None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:15.413524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 561/1111] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:15.419567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:15.420040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:15.420170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:15.442017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:15.443179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:15.504809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 551/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:15.528161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9f58b-7030-452a-9394-9657cb7a169d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:15.623322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9f58b-7030-452a-9394-9657cb7a169d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:15.654827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:15.655647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:15.655923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:15.656193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] PATH_INFO: `/v3/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:15.657921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] RBAC: Authorizing `identity:get_role(role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:15.659252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:15.660571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 562/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:15 2022] GET /identity/v3/roles/d584b6d85205413eb17765aa9c683df4 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:15.666138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:15.762206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713d023-7b55-44d6-96df-a63a67b95f87 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:15.795469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:15.796329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:15.796606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:15.796900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] PATH_INFO: `/v3/users/9b8b06bff1e04c4d91ff45f62e69a039` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:15.809879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] RBAC: Authorizing `identity:get_user(user_id=9b8b06bff1e04c4d91ff45f62e69a039)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:15.811294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:15.812928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 552/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:15 2022] GET /identity/v3/users/9b8b06bff1e04c4d91ff45f62e69a039 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:15.818647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:15.912059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83504b7c-7987-4256-9c66-a663f94447d2 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:15.946536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:15.947109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:15.947390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:15.947666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:16.012231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:16.013703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:16.108961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 563/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:16.114795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:16.206430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d96a8-e352-4b56-b45c-4e78b23a601f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:16.237417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d7da3-fbdf-4ecf-81fa-ff44cc272208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:16.238244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d7da3-fbdf-4ecf-81fa-ff44cc272208 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:16.238520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d7da3-fbdf-4ecf-81fa-ff44cc272208 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:16.238791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d7da3-fbdf-4ecf-81fa-ff44cc272208 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:16.239931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d7da3-fbdf-4ecf-81fa-ff44cc272208 None admin] RBAC: Authorizing `identity:get_project(project_id=dda24e6463a545ed985165c26beb467c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:16.241356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d7da3-fbdf-4ecf-81fa-ff44cc272208 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:16.242974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 553/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:16 2022] GET /identity/v3/projects/dda24e6463a545ed985165c26beb467c => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:16.249654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:16.341733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9ba26-083d-46c2-b8b6-584d7f6f094e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:16.373312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:16.373940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:16.374224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:16.374499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:16.375313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] RBAC: Adding query filter params (role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=dda24e6463a545ed985165c26beb467c, user.id=9b8b06bff1e04c4d91ff45f62e69a039) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:16.375626 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:16.377069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400f288-a879-4fa3-9ee5-9f984f30bc65 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:16.382195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 564/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:16 2022] GET /identity/v3/role_assignments?user.id=9b8b06bff1e04c4d91ff45f62e69a039&scope.project.id=dda24e6463a545ed985165c26beb467c&role.id=d584b6d85205413eb17765aa9c683df4 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:18.538141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89422bd2-68a2-42d3-ac70-76569f199f59 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:18.538667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89422bd2-68a2-42d3-ac70-76569f199f59 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:18.538667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89422bd2-68a2-42d3-ac70-76569f199f59 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:18.540000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 554/1118] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:18.545832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:18.546242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:18.546391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:18.563172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:18.564344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:18.626710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 565/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:18.648232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89422bd2-68a2-42d3-ac70-76569f199f59 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:18.742017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89422bd2-68a2-42d3-ac70-76569f199f59 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:18.772473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:18.773316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:18.773595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:18.773866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] PATH_INFO: `/v3/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:18.775287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] RBAC: Authorizing `identity:get_role(role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:18.777159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:18.778470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 555/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:18 2022] GET /identity/v3/roles/d584b6d85205413eb17765aa9c683df4 => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:18.784398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:18.875926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3f2d3-6244-44c0-afd4-b40b61fc44e5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:18.908469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:18.909352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:18.909634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:18.909906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] PATH_INFO: `/v3/users/9b8b06bff1e04c4d91ff45f62e69a039` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:18.911081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] RBAC: Authorizing `identity:get_user(user_id=9b8b06bff1e04c4d91ff45f62e69a039)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:18.912497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:18.914140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 566/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:18 2022] GET /identity/v3/users/9b8b06bff1e04c4d91ff45f62e69a039 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:18.919613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:19.011576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3821c91-12c6-43cd-9015-58d433811e94 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:19.042978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad998b-b9fa-4dc7-8461-d0bdd5e259d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:19.043838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad998b-b9fa-4dc7-8461-d0bdd5e259d3 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:19.044348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad998b-b9fa-4dc7-8461-d0bdd5e259d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:19.044671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad998b-b9fa-4dc7-8461-d0bdd5e259d3 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:19.046003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad998b-b9fa-4dc7-8461-d0bdd5e259d3 None admin] RBAC: Authorizing `identity:get_project(project_id=dda24e6463a545ed985165c26beb467c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:19.047407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad998b-b9fa-4dc7-8461-d0bdd5e259d3 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:19.049090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 556/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:18 2022] GET /identity/v3/projects/dda24e6463a545ed985165c26beb467c => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:19.054892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:19.147618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41465794-24fa-4c61-b630-1df2b944d24d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:19.179664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34f5df-b938-432b-9bdf-092f6732e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:19.180532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34f5df-b938-432b-9bdf-092f6732e0b3 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:19.180846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34f5df-b938-432b-9bdf-092f6732e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:19.181122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34f5df-b938-432b-9bdf-092f6732e0b3 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c/users/9b8b06bff1e04c4d91ff45f62e69a039/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:19.183323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34f5df-b938-432b-9bdf-092f6732e0b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda24e6463a545ed985165c26beb467c, user_id=9b8b06bff1e04c4d91ff45f62e69a039, role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:19.184772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34f5df-b938-432b-9bdf-092f6732e0b3 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:19.197088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 567/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:39:19 2022] PUT /identity/v3/projects/dda24e6463a545ed985165c26beb467c/users/9b8b06bff1e04c4d91ff45f62e69a039/roles/d584b6d85205413eb17765aa9c683df4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:39:21.366926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da661d12-6851-4417-be70-5741fe76a771 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:21.367449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da661d12-6851-4417-be70-5741fe76a771 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:21.367449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da661d12-6851-4417-be70-5741fe76a771 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:21.368844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 557/1124] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:21.374656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:21.375104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:21.375330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:21.391755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:21.392950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:21.455105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 568/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:21.478317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da661d12-6851-4417-be70-5741fe76a771 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:21.571964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da661d12-6851-4417-be70-5741fe76a771 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:21.603092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:21.604501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:21.604819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:21.605092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] PATH_INFO: `/v3/roles/d584b6d85205413eb17765aa9c683df4` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:21.606520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] RBAC: Authorizing `identity:get_role(role_id=d584b6d85205413eb17765aa9c683df4)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:21.607854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:21.609183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 558/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:21 2022] GET /identity/v3/roles/d584b6d85205413eb17765aa9c683df4 => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:21.614649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:21.706252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311d035-9254-44cd-a6ab-7e81486a3240 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:21.738158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:21.739004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:21.739278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:21.739548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] PATH_INFO: `/v3/users/9b8b06bff1e04c4d91ff45f62e69a039` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:21.740737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] RBAC: Authorizing `identity:get_user(user_id=9b8b06bff1e04c4d91ff45f62e69a039)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:21.742155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:21.743737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 569/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:21 2022] GET /identity/v3/users/9b8b06bff1e04c4d91ff45f62e69a039 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:21.749228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:21.841257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc3db-be12-4c31-a06e-997e7289196c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:21.872697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:21.873318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:21.873602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:21.873875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:21.936316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:21.937784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:22.033180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 559/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:22.038965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:22.131182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44603ed7-f7b4-4663-b96c-d407844d1659 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:22.163190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02c83c-811b-49f8-86b7-fed0d286c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:22.164022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c83c-811b-49f8-86b7-fed0d286c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:22.164300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c83c-811b-49f8-86b7-fed0d286c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:22.164570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c83c-811b-49f8-86b7-fed0d286c2c9 None admin] PATH_INFO: `/v3/projects/dda24e6463a545ed985165c26beb467c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:22.165744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02c83c-811b-49f8-86b7-fed0d286c2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=dda24e6463a545ed985165c26beb467c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:22.167186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02c83c-811b-49f8-86b7-fed0d286c2c9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:22.168873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 570/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:22 2022] GET /identity/v3/projects/dda24e6463a545ed985165c26beb467c => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:22.174755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:22.265465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ee516-60cd-47eb-aa96-e025357facfc None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:22.295995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:22.296613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:22.296918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:22.297188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:22.297989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] RBAC: Adding query filter params (role.id=d584b6d85205413eb17765aa9c683df4, scope.project.id=dda24e6463a545ed985165c26beb467c, user.id=9b8b06bff1e04c4d91ff45f62e69a039) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:22.298294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:22.299678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fea450-d581-42b0-ad92-01c61ae470f8 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:22.305099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 560/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:22 2022] GET /identity/v3/role_assignments?user.id=9b8b06bff1e04c4d91ff45f62e69a039&scope.project.id=dda24e6463a545ed985165c26beb467c&role.id=d584b6d85205413eb17765aa9c683df4 => generated 608 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:24.528296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba5da3-7e1a-40d8-b149-901ae8374c78 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:24.528890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba5da3-7e1a-40d8-b149-901ae8374c78 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:24.528890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba5da3-7e1a-40d8-b149-901ae8374c78 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:24.530250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 571/1131] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:24.535785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:24.536212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:24.536319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:24.553023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:24.554196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:24.615380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 561/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:24.638269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba5da3-7e1a-40d8-b149-901ae8374c78 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:24.734559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba5da3-7e1a-40d8-b149-901ae8374c78 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:24.766001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a91399-8402-450b-b79f-589b5ee41229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:24.766774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91399-8402-450b-b79f-589b5ee41229 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:24.767048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91399-8402-450b-b79f-589b5ee41229 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:24.767316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91399-8402-450b-b79f-589b5ee41229 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:24.768468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a91399-8402-450b-b79f-589b5ee41229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:24.769930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a91399-8402-450b-b79f-589b5ee41229 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:24.771501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 572/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:24.778535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:24.870220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e7d55-9499-47b2-b7d4-f26089395410 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:24.902391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6c73c-3a92-4116-8703-cb8b00f8b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:24.903034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c73c-3a92-4116-8703-cb8b00f8b1f2 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:24.903314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c73c-3a92-4116-8703-cb8b00f8b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:24.903588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c73c-3a92-4116-8703-cb8b00f8b1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:24.904295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c73c-3a92-4116-8703-cb8b00f8b1f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:24.905784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c73c-3a92-4116-8703-cb8b00f8b1f2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:24.929398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 562/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:39:24 2022] POST /identity/v3/projects => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:39:27.144458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3668-838b-4565-932e-8b5cce3d9856 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:27.145043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3668-838b-4565-932e-8b5cce3d9856 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:27.145043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a3668-838b-4565-932e-8b5cce3d9856 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:27.146404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 573/1135] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:27.152543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:27.153003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:27.153104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:27.169498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:27.170662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:27.231746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 563/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:27.256294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a3668-838b-4565-932e-8b5cce3d9856 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:27.352397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a3668-838b-4565-932e-8b5cce3d9856 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:27.383230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a5573-8593-4879-a76a-dd7da3090fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:27.383990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5573-8593-4879-a76a-dd7da3090fa0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:27.384264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5573-8593-4879-a76a-dd7da3090fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:27.384531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5573-8593-4879-a76a-dd7da3090fa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:27.386196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a5573-8593-4879-a76a-dd7da3090fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:27.387597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a5573-8593-4879-a76a-dd7da3090fa0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:27.389192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 574/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:39:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:27.394511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:27.485142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e7a5e-ebc2-4278-88e5-00e24d72d4a9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:27.516965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9b7b1-f56a-40ba-8945-4be4d6acba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:27.517612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9b7b1-f56a-40ba-8945-4be4d6acba65 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:27.517886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9b7b1-f56a-40ba-8945-4be4d6acba65 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:27.518154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9b7b1-f56a-40ba-8945-4be4d6acba65 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:27.518850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9b7b1-f56a-40ba-8945-4be4d6acba65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:27.520260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9b7b1-f56a-40ba-8945-4be4d6acba65 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:27.538910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 564/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:39:27 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:39:29.737733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71628b0-40a5-4cc1-8ac5-2ff759f3c060 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:29.738262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71628b0-40a5-4cc1-8ac5-2ff759f3c060 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:29.738262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71628b0-40a5-4cc1-8ac5-2ff759f3c060 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:29.739628 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 575/1139] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:29.745511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:29.745909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:29.746053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:29.762859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:29.764022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:29.832426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 565/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:29.856561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71628b0-40a5-4cc1-8ac5-2ff759f3c060 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:29.951672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71628b0-40a5-4cc1-8ac5-2ff759f3c060 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:29.982752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:29.983521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:29.983804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:29.984076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:29.993546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:30.001563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:30.002738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:30.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 576/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:30.008597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:30.099697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be70710-25db-4b1a-9453-299cbf163886 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:30.141685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:30.142809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:30.143307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:30.143808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:30.145208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:30.145773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:30.147503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:30.159106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 566/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:30.165054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:30.262544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeebc18-47ee-4eaf-aca0-f4bbf37fad81 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:30.293777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:30.294624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:30.294903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:30.295173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] PATH_INFO: `/v3/users/8446e599004c4a6f98019877761424b2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:30.308052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] RBAC: Authorizing `identity:get_user(user_id=8446e599004c4a6f98019877761424b2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:30.309497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:30.311101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 577/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:30 2022] GET /identity/v3/users/8446e599004c4a6f98019877761424b2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:30.316847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:30.408127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db7bb7-d72c-421e-be85-8f36ae5bbd33 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:30.439430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:30.440046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:30.440324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:30.440598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:30.509689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:30.511153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:30.601678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 567/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:30.608271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:30.700280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d931744-4a09-4958-b775-36da406c87e8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:30.734423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699de8d-2bc9-4673-83ac-24dce62e6b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:30.735262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699de8d-2bc9-4673-83ac-24dce62e6b4e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:30.735543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699de8d-2bc9-4673-83ac-24dce62e6b4e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:30.735816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699de8d-2bc9-4673-83ac-24dce62e6b4e None admin] PATH_INFO: `/v3/projects/58baf1ab9f9e41849c5e42e59631643e` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:30.736999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699de8d-2bc9-4673-83ac-24dce62e6b4e None admin] RBAC: Authorizing `identity:get_project(project_id=58baf1ab9f9e41849c5e42e59631643e)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:30.738410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699de8d-2bc9-4673-83ac-24dce62e6b4e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:30.740080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 578/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:30 2022] GET /identity/v3/projects/58baf1ab9f9e41849c5e42e59631643e => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:30.745987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:30.846372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18a2a-24e7-466b-883d-151eaf1b329a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:30.878126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:30.878705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:30.878986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:30.879258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:30.880069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=58baf1ab9f9e41849c5e42e59631643e, user.id=8446e599004c4a6f98019877761424b2) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:30.880379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:30.881819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ff6fd-7340-43b5-abb1-0997d9a9d139 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:30.887484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 568/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:30 2022] GET /identity/v3/role_assignments?user.id=8446e599004c4a6f98019877761424b2&scope.project.id=58baf1ab9f9e41849c5e42e59631643e&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:33.042853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b2a7-f546-4334-8782-52ba437d92c2 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.043346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b2a7-f546-4334-8782-52ba437d92c2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.043439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b2a7-f546-4334-8782-52ba437d92c2 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.044787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 579/1147] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:33.050005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.050423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.050559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.067151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:33.068309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:33.130899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 569/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:33.153502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310b2a7-f546-4334-8782-52ba437d92c2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:33.248682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310b2a7-f546-4334-8782-52ba437d92c2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:33.279884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:33.280668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.280948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.281216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.291345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:33.298789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:33.299838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:33.300742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 580/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:33.306035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:33.397026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39ebad-9f07-4be4-a55d-05d4f124aee7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:33.428693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:33.429338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.429618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.429891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.430624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:33.430931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:33.432855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:33.440344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 570/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:33.446179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:33.538291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1547-4f08-486f-b895-495f85c6c59e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:33.569698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:33.570526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.570801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.571069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] PATH_INFO: `/v3/users/8446e599004c4a6f98019877761424b2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.573043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] RBAC: Authorizing `identity:get_user(user_id=8446e599004c4a6f98019877761424b2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:33.574538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:33.576191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 581/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:33 2022] GET /identity/v3/users/8446e599004c4a6f98019877761424b2 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:33.581784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:33.673832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6a8cc-1c66-493a-aca7-59177d24a13b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:33.705256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e6c93-03e8-4c89-82e3-66dc7312d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:33.706088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e6c93-03e8-4c89-82e3-66dc7312d1dd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.706369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e6c93-03e8-4c89-82e3-66dc7312d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.706641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e6c93-03e8-4c89-82e3-66dc7312d1dd None admin] PATH_INFO: `/v3/projects/58baf1ab9f9e41849c5e42e59631643e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.707787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e6c93-03e8-4c89-82e3-66dc7312d1dd None admin] RBAC: Authorizing `identity:get_project(project_id=58baf1ab9f9e41849c5e42e59631643e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:33.709224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e6c93-03e8-4c89-82e3-66dc7312d1dd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:33.710848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 571/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:33 2022] GET /identity/v3/projects/58baf1ab9f9e41849c5e42e59631643e => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:33.716417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:33.808886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d5d2-2dc4-4ef7-a570-5d2a28338aea None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:33.840013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695505-3da8-461e-82c5-550f2a0a5fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:33.840911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695505-3da8-461e-82c5-550f2a0a5fc6 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:33.841189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695505-3da8-461e-82c5-550f2a0a5fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:33.841458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695505-3da8-461e-82c5-550f2a0a5fc6 None admin] PATH_INFO: `/v3/projects/58baf1ab9f9e41849c5e42e59631643e/users/8446e599004c4a6f98019877761424b2/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:33.843393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695505-3da8-461e-82c5-550f2a0a5fc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=58baf1ab9f9e41849c5e42e59631643e, user_id=8446e599004c4a6f98019877761424b2, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:33.844831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695505-3da8-461e-82c5-550f2a0a5fc6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:33.857055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 582/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:39:33 2022] PUT /identity/v3/projects/58baf1ab9f9e41849c5e42e59631643e/users/8446e599004c4a6f98019877761424b2/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:39:36.011776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf437b7-8357-4dd1-bfb7-c08ec64e2cc0 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.012268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf437b7-8357-4dd1-bfb7-c08ec64e2cc0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.012360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf437b7-8357-4dd1-bfb7-c08ec64e2cc0 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.013690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 572/1154] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:36.019027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.019442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.019559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.036441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:36.037627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:36.098797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 583/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:36.122195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf437b7-8357-4dd1-bfb7-c08ec64e2cc0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:36.218492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf437b7-8357-4dd1-bfb7-c08ec64e2cc0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:36.249134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:36.249901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.250182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.250456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.260903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:36.268378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:36.269475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:36.270667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 573/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:36.276175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:36.366667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feeb8f-7d9c-468b-9f63-5b3d7cc45c98 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:36.397701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:36.398341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.398618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.398889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.399612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:36.399914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:36.401886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:36.409630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 584/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:36.414970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:36.506805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54071bdd-d2c4-4df2-918c-d56a5daa9168 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:36.538417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:36.539255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.539534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.539805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] PATH_INFO: `/v3/users/8446e599004c4a6f98019877761424b2` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.540986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] RBAC: Authorizing `identity:get_user(user_id=8446e599004c4a6f98019877761424b2)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:36.542394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:36.543954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 574/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:36 2022] GET /identity/v3/users/8446e599004c4a6f98019877761424b2 => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:36.549424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:36.641897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddec9e-bd05-4e99-b3f7-fe3572b0ed7f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:36.672708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:36.673329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.673608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.673881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.735609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:36.737065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:36.828554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 585/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:36.834193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:36.927118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41486a-9b01-4fef-8e08-5c6e927aeb4f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:36.959945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21094fb5-4982-4bbe-9013-a7f50bb0a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:36.960484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21094fb5-4982-4bbe-9013-a7f50bb0a75f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:36.960860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21094fb5-4982-4bbe-9013-a7f50bb0a75f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:36.961275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21094fb5-4982-4bbe-9013-a7f50bb0a75f None admin] PATH_INFO: `/v3/projects/58baf1ab9f9e41849c5e42e59631643e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:36.963073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21094fb5-4982-4bbe-9013-a7f50bb0a75f None admin] RBAC: Authorizing `identity:get_project(project_id=58baf1ab9f9e41849c5e42e59631643e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:36.965420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21094fb5-4982-4bbe-9013-a7f50bb0a75f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:36.967208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 575/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:36 2022] GET /identity/v3/projects/58baf1ab9f9e41849c5e42e59631643e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:36.973825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:37.066338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531ab7-cc41-4249-8864-03e6f6efa1a4 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:37.097370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:37.097939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:37.098216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:37.098487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:37.099290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=58baf1ab9f9e41849c5e42e59631643e, user.id=8446e599004c4a6f98019877761424b2) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:37.099597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:37.101038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98939b95-5b36-4215-af53-54d7fdb02962 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:37.107030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 586/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:36 2022] GET /identity/v3/role_assignments?user.id=8446e599004c4a6f98019877761424b2&scope.project.id=58baf1ab9f9e41849c5e42e59631643e&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:39.330811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958da00-7f21-4ca8-ac2d-75fa90cf20d1 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:39.331358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958da00-7f21-4ca8-ac2d-75fa90cf20d1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:39.331358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958da00-7f21-4ca8-ac2d-75fa90cf20d1 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:39.332795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 576/1162] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:39.338657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:39.339082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:39.339190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:39.355198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:39.356369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:39.420494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 587/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:39.442696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958da00-7f21-4ca8-ac2d-75fa90cf20d1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:39.538457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958da00-7f21-4ca8-ac2d-75fa90cf20d1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:39.570073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:39.570693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:39.570969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:39.571238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:39.635067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:39.636456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:39.728868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 577/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:39:39.734855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:39.832429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075cfa8-41c8-4c86-b2ed-230eb4880edf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:39.864932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:39.866027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:39.866510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:39.866995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:39.874171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:39.875600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-46e9bd87-6947-4868-be95-f2829f957748 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:39.891069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:39.892237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:39.893196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 588/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:39:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:39.898431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:40.012149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0bf26-522f-4a9d-bfe2-0d8f26c39cfa None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:40.044325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:40.045003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:40.045286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:40.045563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:40.046282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:40.046875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:40.048459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685deb9d-a412-4ae7-9540-1a6eb491db5f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:40.055549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 578/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 16 18:39:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:42.220291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cdf5-14c1-4293-bc03-8d56a49f9fd5 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:42.220887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cdf5-14c1-4293-bc03-8d56a49f9fd5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:42.220887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cdf5-14c1-4293-bc03-8d56a49f9fd5 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:42.222149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 589/1167] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:42.227402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fa1d7-8c93-4cbe-a130-839d12a50c04 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:42.227791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fa1d7-8c93-4cbe-a130-839d12a50c04 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:42.227963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fa1d7-8c93-4cbe-a130-839d12a50c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:42.244047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-692fa1d7-8c93-4cbe-a130-839d12a50c04 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:42.245243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-692fa1d7-8c93-4cbe-a130-839d12a50c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:42.307591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 579/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:42.330260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9cdf5-14c1-4293-bc03-8d56a49f9fd5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:42.425145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9cdf5-14c1-4293-bc03-8d56a49f9fd5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:42.456278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761ef74-f5bd-4c98-80a9-74f8896a58bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:42.456946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ef74-f5bd-4c98-80a9-74f8896a58bb None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:42.457227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ef74-f5bd-4c98-80a9-74f8896a58bb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:42.457500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ef74-f5bd-4c98-80a9-74f8896a58bb None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:42.458217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761ef74-f5bd-4c98-80a9-74f8896a58bb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:42.459571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761ef74-f5bd-4c98-80a9-74f8896a58bb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:42.473878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 590/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Mon May 16 18:39:42 2022] POST /identity/v3/domains => generated 267 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:39:44.698706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ceb5a-c350-4e2f-9385-61441bb712e1 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:44.699277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ceb5a-c350-4e2f-9385-61441bb712e1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:44.699277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ceb5a-c350-4e2f-9385-61441bb712e1 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:44.700608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 580/1170] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:44.705800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:44.706215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:44.706348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:44.722376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:44.723555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:44.786140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 591/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:44.809566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ceb5a-c350-4e2f-9385-61441bb712e1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:44.904311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ceb5a-c350-4e2f-9385-61441bb712e1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:44.936771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea33ed-6047-41d3-af62-d63ad62d0692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:44.937614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea33ed-6047-41d3-af62-d63ad62d0692 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:44.937897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea33ed-6047-41d3-af62-d63ad62d0692 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:44.938171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea33ed-6047-41d3-af62-d63ad62d0692 None admin] PATH_INFO: `/v3/domains/d557ed58ab50483487a5bf66ea33f867` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:44.947989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea33ed-6047-41d3-af62-d63ad62d0692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d557ed58ab50483487a5bf66ea33f867)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:44.949502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea33ed-6047-41d3-af62-d63ad62d0692 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:44.951193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 581/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Mon May 16 18:39:44 2022] GET /identity/v3/domains/d557ed58ab50483487a5bf66ea33f867 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:44.956850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:45.048972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9a40d-21d8-4e22-ab19-c6d08dc4f552 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:45.079582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51eb82-adb5-4853-9f5c-d34f5ef67153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:45.080216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51eb82-adb5-4853-9f5c-d34f5ef67153 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:45.080488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51eb82-adb5-4853-9f5c-d34f5ef67153 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:45.080785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51eb82-adb5-4853-9f5c-d34f5ef67153 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:45.081464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51eb82-adb5-4853-9f5c-d34f5ef67153 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:45.082913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51eb82-adb5-4853-9f5c-d34f5ef67153 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:45.107712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 592/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:39:44 2022] POST /identity/v3/projects => generated 361 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:39:47.365021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7500f-c967-4e31-9660-a299f1909f7f None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:47.365567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7500f-c967-4e31-9660-a299f1909f7f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:47.365567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7500f-c967-4e31-9660-a299f1909f7f None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:47.366912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 582/1174] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:47.372033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:47.372485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:47.372591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:47.389377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:47.390541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:47.453458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 593/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:39:47.478096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7500f-c967-4e31-9660-a299f1909f7f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:47.573372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7500f-c967-4e31-9660-a299f1909f7f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:47.604235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7d35e-c3ed-470c-8ba5-43666a12f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:47.605102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d35e-c3ed-470c-8ba5-43666a12f11c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:47.605381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d35e-c3ed-470c-8ba5-43666a12f11c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:47.605653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d35e-c3ed-470c-8ba5-43666a12f11c None admin] PATH_INFO: `/v3/domains/d557ed58ab50483487a5bf66ea33f867` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:47.606807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7d35e-c3ed-470c-8ba5-43666a12f11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=d557ed58ab50483487a5bf66ea33f867)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:47.608219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7d35e-c3ed-470c-8ba5-43666a12f11c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:47.609815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 583/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Mon May 16 18:39:47 2022] GET /identity/v3/domains/d557ed58ab50483487a5bf66ea33f867 => generated 267 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:47.615413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:47.706768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87574f76-f7c7-40ec-a45f-40d6ed6b7bb2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:47.738368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5740e81-6f89-4796-b6ae-89c388b4b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:47.739009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5740e81-6f89-4796-b6ae-89c388b4b2fd None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:47.739289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5740e81-6f89-4796-b6ae-89c388b4b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:47.739558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5740e81-6f89-4796-b6ae-89c388b4b2fd None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:47.740250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5740e81-6f89-4796-b6ae-89c388b4b2fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:47.741693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5740e81-6f89-4796-b6ae-89c388b4b2fd None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:47.761080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 594/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:39:47 2022] POST /identity/v3/users => generated 311 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:39:49.989044 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d417c-231d-428b-b5df-396d6d0b176c None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:49.989485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d417c-231d-428b-b5df-396d6d0b176c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:49.989577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d417c-231d-428b-b5df-396d6d0b176c None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:49.990920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 584/1178] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:49.996572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:49.997042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:49.997144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:50.013910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:50.015062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:50.077631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 595/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:50.099858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d417c-231d-428b-b5df-396d6d0b176c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:50.197446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d417c-231d-428b-b5df-396d6d0b176c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:50.227731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:50.228437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:50.228743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:50.229015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:50.238857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:50.246788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:50.247955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:50.248929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 585/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:50.253842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:50.344874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be9f88-385a-4d7b-9b0e-c0588bc10710 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:50.376051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:50.376712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:50.376991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:50.377267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:50.377990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:50.378288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:50.379630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:50.387684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 596/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:50.393103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:50.484990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84518841-4afb-487a-97a5-6c522b4b44d4 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:50.516341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:50.517217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:50.517499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:50.517771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] PATH_INFO: `/v3/users/6bb77573e71e40ceb050ecb29ba530b0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:50.529848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] RBAC: Authorizing `identity:get_user(user_id=6bb77573e71e40ceb050ecb29ba530b0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:50.531268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:50.533161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 586/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:50 2022] GET /identity/v3/users/6bb77573e71e40ceb050ecb29ba530b0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:50.538952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:50.632337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231c8b0-ed28-404d-81bd-76ed9aec85d7 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:50.662947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:50.663570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:50.663848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:50.664117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:50.725745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:50.727183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:50.818192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 597/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:39:50.824024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:50.916501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a805-7075-48d2-b991-209b58b5dd1a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:50.947492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0cea1-ce78-46fd-bdf4-361252b83cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:50.948326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0cea1-ce78-46fd-bdf4-361252b83cab None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:50.948605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0cea1-ce78-46fd-bdf4-361252b83cab None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:50.948916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0cea1-ce78-46fd-bdf4-361252b83cab None admin] PATH_INFO: `/v3/projects/99c63d153d1b4befbb5a62398fee60ec` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:50.950068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0cea1-ce78-46fd-bdf4-361252b83cab None admin] RBAC: Authorizing `identity:get_project(project_id=99c63d153d1b4befbb5a62398fee60ec)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:50.951474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0cea1-ce78-46fd-bdf4-361252b83cab None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:50.953147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 587/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:50 2022] GET /identity/v3/projects/99c63d153d1b4befbb5a62398fee60ec => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:50.959107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:51.051986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5ea51-1a71-4e52-a5c6-237099be0396 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:51.082876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:51.083448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:51.083723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:51.083991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:51.084832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=99c63d153d1b4befbb5a62398fee60ec, user.id=6bb77573e71e40ceb050ecb29ba530b0) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:51.085137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:51.086537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0b7c8-2703-4c5d-9b52-0931837caadc None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:51.092143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 598/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:50 2022] GET /identity/v3/role_assignments?user.id=6bb77573e71e40ceb050ecb29ba530b0&scope.project.id=99c63d153d1b4befbb5a62398fee60ec&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:39:53.268963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74702b-1e92-408e-a8d6-a38b7a921dca None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:53.269456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74702b-1e92-408e-a8d6-a38b7a921dca None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:53.269456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74702b-1e92-408e-a8d6-a38b7a921dca None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:53.270771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 588/1186] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:53.275805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:53.276224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:53.276353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:53.292764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:53.293933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:53.355719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 599/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:53.378210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74702b-1e92-408e-a8d6-a38b7a921dca None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:53.472236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74702b-1e92-408e-a8d6-a38b7a921dca None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:53.504200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:53.504996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:53.505278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:53.505548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:53.514957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:53.522475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:53.523560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:39:53.524575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 589/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:53.529554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:53.634786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c2ba8-e340-409d-beeb-0ac74e1b2e7d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:53.667213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:53.667864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:53.668143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:53.668413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:53.669167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:53.669467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:53.670821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:53.678400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 600/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:53.684988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:53.778498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9083a-ffc3-47aa-9e78-3c92ef113aab None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:53.809641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:53.810472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:53.810749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:53.811019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] PATH_INFO: `/v3/users/6bb77573e71e40ceb050ecb29ba530b0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:53.812175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] RBAC: Authorizing `identity:get_user(user_id=6bb77573e71e40ceb050ecb29ba530b0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:53.813623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:53.815185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 590/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:53 2022] GET /identity/v3/users/6bb77573e71e40ceb050ecb29ba530b0 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:53.820804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:53.913389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c80f1-4568-457a-b28b-f5f7499ef376 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:53.944936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b7dc4-07ea-4035-a8d7-dab4d6a98c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:53.945774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7dc4-07ea-4035-a8d7-dab4d6a98c86 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:53.946047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7dc4-07ea-4035-a8d7-dab4d6a98c86 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:53.946312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b7dc4-07ea-4035-a8d7-dab4d6a98c86 None admin] PATH_INFO: `/v3/projects/99c63d153d1b4befbb5a62398fee60ec` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:53.947459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b7dc4-07ea-4035-a8d7-dab4d6a98c86 None admin] RBAC: Authorizing `identity:get_project(project_id=99c63d153d1b4befbb5a62398fee60ec)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:53.948900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b7dc4-07ea-4035-a8d7-dab4d6a98c86 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:53.950577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 601/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:53 2022] GET /identity/v3/projects/99c63d153d1b4befbb5a62398fee60ec => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:53.956659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:54.051565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9936313-dd8d-405b-a22c-f098a941ed5b None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:54.081671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4f25-145f-4e0c-848d-3b2508e9b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:54.082527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4f25-145f-4e0c-848d-3b2508e9b705 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:54.082802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4f25-145f-4e0c-848d-3b2508e9b705 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:54.083067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4f25-145f-4e0c-848d-3b2508e9b705 None admin] PATH_INFO: `/v3/projects/99c63d153d1b4befbb5a62398fee60ec/users/6bb77573e71e40ceb050ecb29ba530b0/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:54.084990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4f25-145f-4e0c-848d-3b2508e9b705 None admin] RBAC: Authorizing `identity:create_grant(project_id=99c63d153d1b4befbb5a62398fee60ec, user_id=6bb77573e71e40ceb050ecb29ba530b0, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:54.086389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4f25-145f-4e0c-848d-3b2508e9b705 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:54.098995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 591/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:39:53 2022] PUT /identity/v3/projects/99c63d153d1b4befbb5a62398fee60ec/users/6bb77573e71e40ceb050ecb29ba530b0/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:39:56.263981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496565-caea-483e-b2b7-d208f41ad4f6 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:56.264495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496565-caea-483e-b2b7-d208f41ad4f6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:56.264495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496565-caea-483e-b2b7-d208f41ad4f6 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:56.266018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 602/1193] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:39:56.270926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:56.271320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:56.271475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:56.287617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:39:56.288804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:39:56.349556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 592/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:39:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:39:56.372278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496565-caea-483e-b2b7-d208f41ad4f6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:56.468200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496565-caea-483e-b2b7-d208f41ad4f6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:56.499999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:56.500790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:56.501072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:56.501342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:56.510738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:39:56.518266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:39:56.519345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:39:56.520529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 603/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:39:56.525210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:56.615808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898ac8b-b41f-4103-bb56-ac053a15ff7c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:56.646935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:56.647579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:56.647859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:56.648131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:56.648889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:56.649275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:56.651685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:56.662687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 593/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:56.669018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:56.762768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e4964-6731-4fb1-88b3-1746fe2a8d3b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:56.794296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:56.795131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:56.795405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:56.795673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] PATH_INFO: `/v3/users/6bb77573e71e40ceb050ecb29ba530b0` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:56.796842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] RBAC: Authorizing `identity:get_user(user_id=6bb77573e71e40ceb050ecb29ba530b0)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:56.798250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:56.799831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 604/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 16 18:39:56 2022] GET /identity/v3/users/6bb77573e71e40ceb050ecb29ba530b0 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:56.805653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:56.896027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367f4b9-8550-4cce-84f3-f061b9c633be None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:56.927567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:56.928187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:56.928462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:56.928773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:56.995238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:56.996688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:57.087693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 594/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:39:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:39:57.093478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:57.185843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58aaf-5926-4e02-92cd-251b7edbb391 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:57.216960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7a915-1e48-4e9f-8f2f-fb791aa47fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:57.217788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7a915-1e48-4e9f-8f2f-fb791aa47fbb None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:57.218064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7a915-1e48-4e9f-8f2f-fb791aa47fbb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:57.218333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7a915-1e48-4e9f-8f2f-fb791aa47fbb None admin] PATH_INFO: `/v3/projects/99c63d153d1b4befbb5a62398fee60ec` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:57.219461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7a915-1e48-4e9f-8f2f-fb791aa47fbb None admin] RBAC: Authorizing `identity:get_project(project_id=99c63d153d1b4befbb5a62398fee60ec)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:57.220920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7a915-1e48-4e9f-8f2f-fb791aa47fbb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:57.222550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 605/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 16 18:39:57 2022] GET /identity/v3/projects/99c63d153d1b4befbb5a62398fee60ec => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:39:57.228308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:39:57.325350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100574c-d21f-4747-97bd-f896d8f15b30 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:39:57.356907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:39:57.357485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:39:57.357763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:39:57.358034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:39:57.358839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=99c63d153d1b4befbb5a62398fee60ec, user.id=6bb77573e71e40ceb050ecb29ba530b0) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:39:57.359142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:39:57.360554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77fc1e-be9f-475c-95d1-0c45236a6090 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:39:57.366087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 595/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:39:57 2022] GET /identity/v3/role_assignments?user.id=6bb77573e71e40ceb050ecb29ba530b0&scope.project.id=99c63d153d1b4befbb5a62398fee60ec&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:40:57.408596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da27a-a0d4-4ff5-a88a-7e5ccd28478b None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:40:57.409394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da27a-a0d4-4ff5-a88a-7e5ccd28478b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:40:57.409394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da27a-a0d4-4ff5-a88a-7e5ccd28478b None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:40:57.410979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 606/1201] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:40:57.416712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:40:57.417185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:40:57.417271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:40:57.439119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:40:57.440526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:40:57.525244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 596/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:40:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 16 18:40:57.550211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59da27a-a0d4-4ff5-a88a-7e5ccd28478b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:40:57.674024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59da27a-a0d4-4ff5-a88a-7e5ccd28478b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:40:57.719978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:40:57.720927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:40:57.721220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:40:57.721500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:40:57.736413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:40:57.738011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:40:57.759798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:40:57.761108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:40:57.762350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:40:57.762350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:40:57.762350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:40:57.762350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:40:57.762350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 607/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:40:57.768767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:40:57.890622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfcfe7-4c5e-4ed7-ad8c-b2a16bd651bf None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:40:57.938057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:40:57.939347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:40:57.939630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:40:57.940485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:40:57.941183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:40:57.941183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:40:57.942642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f808f-e77e-47e8-b2b0-f0431b0cc39e None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:40:57.957323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 597/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:40:57.962575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:40:58.089150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f426ed6-2c5f-4e01-878f-6a443f72595d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:40:58.128832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bded7a-92b8-4fa8-8378-a48745f19acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:40:58.129563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bded7a-92b8-4fa8-8378-a48745f19acb None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:40:58.129941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bded7a-92b8-4fa8-8378-a48745f19acb None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:40:58.130343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bded7a-92b8-4fa8-8378-a48745f19acb None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:40:58.131702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bded7a-92b8-4fa8-8378-a48745f19acb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:40:58.133440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bded7a-92b8-4fa8-8378-a48745f19acb None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:40:58.164863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 608/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 16 18:40:57 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:41:00.720673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c385a4-bc0e-4fd2-8081-050094d4bfe6 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:00.720673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c385a4-bc0e-4fd2-8081-050094d4bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:00.720673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c385a4-bc0e-4fd2-8081-050094d4bfe6 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:00.721940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 598/1206] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:00.728313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:00.728821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:00.728961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:00.746717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:00.747889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:00.821822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 609/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:00.846759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c385a4-bc0e-4fd2-8081-050094d4bfe6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:00.958355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c385a4-bc0e-4fd2-8081-050094d4bfe6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:00.996260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:00.997152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:00.997547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:00.997844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:01.010382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:41:01.018019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:01.019162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:01.020005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 599/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:41:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:01.025990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:01.128601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec64482-ea71-4e2a-af83-c1e4ee4447e6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:01.163789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:01.164511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:01.164805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:01.165079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:01.165825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:01.166122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:01.167556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:01.176496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 610/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:41:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:01.186012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:01.294129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f4cc5-0407-4be4-8228-8636a1003f64 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:01.331033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:01.331888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:01.332169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:01.332542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:01.343337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:01.344867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:01.362341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:01.363343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:01.369948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:01.369948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:01.369948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:01.369948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:01.369948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 600/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:01.370245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:01.474871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87f518-232d-4195-a20f-09bd0d488b49 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:01.520038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:01.520731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:01.521007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:01.521276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:01.521997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:01.522302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:01.523656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:01.545046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 611/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:01.547191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:01.649578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977942b-e45b-4df2-aa81-a3294d83573c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:01.686202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:01.687012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:01.687294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:01.687593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:01.695371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:01.696900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:41:01.711544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:01.712555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:01.713546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 601/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:41:01.718561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:01.821719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d132cd-dd9f-4e80-8fe3-915d6019c065 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:01.859640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:01.860305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:01.860585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:01.860933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:01.861760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:01.862065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:01.863535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:01.886340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 612/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 16 18:41:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:01.887212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:01.998594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6343914-e294-4496-aea4-ba50dd79ef57 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:02.042713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:02.043432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:02.043718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:02.044047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:02.124260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:02.125695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:02.240779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 602/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:41:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:41:02.246564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:02.354950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e469076-bf8d-4a7f-8c2f-fa89be07cac1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:02.397062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:02.398054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:02.398341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:02.398718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:02.411180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:02.413483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:02.432154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:02.433169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:02.434607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:02.434607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:02.434607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:02.434607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:02.434607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 613/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:02.439060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:02.550774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95189-0ac3-4be3-bc7e-90a023c1409d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:02.601587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:02.602405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:02.602753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:02.606886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:02.606886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:02.606886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:02.606886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:02.623909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 603/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:02.625478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:02.730744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8bb22-1da3-4c8d-8108-a17cf55d6579 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:02.765459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:02.766300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:02.766581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:02.766839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:02.774413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:02.775977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:41:02.790968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:02.792190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:02.793365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 614/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:02.798484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:02.911065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c2970-1a87-4fe7-bba8-1c9d1264410d None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:02.946761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:02.947477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:02.947772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:02.948085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:02.949948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:02.950267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:02.951711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61fb59-9456-485a-9b0c-0d0fb58ea193 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:02.966383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 604/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:02.972550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:03.084308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65690507-d3e0-4275-9419-d5340562c68d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:03.124384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:03.125289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:03.125666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:03.126053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:03.127171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=c103464c1bca4da29e61fe73b6fd1d92) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:03.127630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:03.129559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f726a09-6297-4566-997c-d38e6ad27718 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:03.144734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 615/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:41:02 2022] GET /identity/v3/role_assignments?user.id=c103464c1bca4da29e61fe73b6fd1d92&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:05.414746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1c3dc-7ec9-4d70-a327-9494f9138abd None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:05.415277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1c3dc-7ec9-4d70-a327-9494f9138abd None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:05.415277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1c3dc-7ec9-4d70-a327-9494f9138abd None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:05.416799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 605/1220] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:41:05.422158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:05.422586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:05.422766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:05.439458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:05.440661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:05.510540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 616/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:05.533109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1c3dc-7ec9-4d70-a327-9494f9138abd None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:05.635842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1c3dc-7ec9-4d70-a327-9494f9138abd None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:05.671682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:05.672393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:05.672697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:05.672971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:05.684017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:41:05.691720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:05.692721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:05.693589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 606/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:05.699329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:05.803201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c07b46-d62e-44d6-a387-623de23c7c8a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:05.838026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:05.838773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:05.839108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:05.839362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:05.840130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] RBAC: Adding query filter params (name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:05.840441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:05.841921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:05.851823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 617/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:41:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:05.858111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:05.962539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f0ad3-77c9-42d5-8e96-dbf81c97997c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:05.996493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:05.997311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:05.997588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:05.997889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:06.008782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:06.010328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:06.029899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:06.031187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:06.032337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:06.032337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:06.032337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:06.032337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:06.032337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 607/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:06.037404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:06.157074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7451-6c0d-4962-a24c-d654ebc39f8c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:06.195316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:06.196809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:06.196809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:06.196809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:06.197498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:06.197884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:06.199296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:06.213773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 618/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:06.219723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:06.329265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2a75d-517e-4b20-8dd2-df89b244c423 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:06.365791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:06.366823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:06.367210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:06.367493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:06.375391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:06.376944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:41:06.391893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:06.393097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:06.394621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 608/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:06.399671 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:06.503859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58706ece-b625-4b64-877a-665611c51f55 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:06.539989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:06.540747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:06.541025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:06.541331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:06.542258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:06.542567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:06.544133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:06.567237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 619/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 16 18:41:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:06.569127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:06.679161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8890baf-592c-490d-8f3c-1d3bd3eeac15 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:06.717810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:06.718636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:06.718954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:06.719261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:06.730878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:06.732828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:06.752106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:06.753254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:06.754530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:06.754530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:06.754530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:06.754530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:06.756666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 609/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:06.763164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:06.870864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5a90-af99-4b4e-95cd-4fccc0a1d76b None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:06.907755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:06.907755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:06.907755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:06.907755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:06.908174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:06.908174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:06.909474 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:06.921143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 620/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:06.927941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:07.034253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d4d78-6807-411d-a8d0-66f5ae459e1a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:07.069958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:07.071009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:07.071323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:07.071616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:07.079298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:07.080964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:41:07.097823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:07.100092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:07.101220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 610/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:07.106228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:07.217146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37739b-9aed-4727-97e0-180fb9a686c8 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:07.254312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:07.254965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:07.255271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:07.255544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:07.256327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:07.256652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:07.258081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a991204-5685-4861-a181-d3e830cbc8d0 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:07.271688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 621/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:07.277880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:07.382952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb66de3-3649-4cb2-8908-b49b1b0d66d1 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:07.420900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dccdcd-e497-43f1-8361-d9af55423242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:07.421842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dccdcd-e497-43f1-8361-d9af55423242 None admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:07.422152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dccdcd-e497-43f1-8361-d9af55423242 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:07.422440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dccdcd-e497-43f1-8361-d9af55423242 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/c103464c1bca4da29e61fe73b6fd1d92/roles/1dd3964ba91e4f5084105ef2d01f66ce` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:07.437801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dccdcd-e497-43f1-8361-d9af55423242 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=c103464c1bca4da29e61fe73b6fd1d92, role_id=1dd3964ba91e4f5084105ef2d01f66ce)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:07.439704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dccdcd-e497-43f1-8361-d9af55423242 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:07.452922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 611/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:41:07 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/c103464c1bca4da29e61fe73b6fd1d92/roles/1dd3964ba91e4f5084105ef2d01f66ce => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:41:09.773996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f619962-72f7-4637-8f0e-451cbe5b3228 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:09.774587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f619962-72f7-4637-8f0e-451cbe5b3228 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:09.774587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f619962-72f7-4637-8f0e-451cbe5b3228 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:09.776256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 622/1233] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:09.782759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:09.783294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:09.783716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:09.803870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:09.805631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:09.877515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 612/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:09.901304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f619962-72f7-4637-8f0e-451cbe5b3228 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:10.004249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f619962-72f7-4637-8f0e-451cbe5b3228 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:10.038471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:10.039227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:10.039509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:10.039780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:10.051258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:41:10.060175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:10.061177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:10.062416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 623/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 16 18:41:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:10.067906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:10.173740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512d779-b0b9-4fcc-b748-667ebd8e34e4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:10.208303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:10.209093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:10.209374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:10.209660 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:10.210851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] RBAC: Adding query filter params (name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:10.211151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:10.212618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:10.222761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 613/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 16 18:41:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:10.229244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:10.337260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08886-e865-4887-bfaa-0727d2edad8d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:10.375654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:10.376441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:10.376750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:10.377024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:10.388755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:10.390369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:10.409213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:10.410196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:10.411826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:10.411826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:10.411826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:10.411826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:10.411826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 624/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:41:10.416781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:10.516534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f38e7b-d2f3-451e-8828-5392294a6b31 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:10.555978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:10.556804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:10.557127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:10.557363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:10.558210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:10.558517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:10.559868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:10.572479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 614/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:10.578269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:10.703200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01624498-14f6-437b-a603-3eed8cb31d1a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:10.737610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:10.738508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:10.738784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:10.739052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:10.746756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:10.748177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:41:10.764475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:10.765605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:10.766547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 625/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:10.772426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:10.872905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e626a-3123-49b1-81ce-ddd8e0ef15e0 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:10.910658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:10.911439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:10.911713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:10.911982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:10.912847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:10.913271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:10.914704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:10.940028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 615/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 16 18:41:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:10.947092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:11.058687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb51b7-b55c-4a68-ab1f-febc00f8ba7a None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:11.099695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:11.101085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:11.101686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:11.102256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:11.187248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:11.188915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:11.305435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 626/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:41:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 16 18:41:11.311282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:11.407412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b280afd-8425-4d63-8730-1d0a51393f1c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:11.441538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:11.442531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:11.442815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:11.443095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:11.453851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:11.455273 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:11.473484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:11.475152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:11.476146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:11.476146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:11.476146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:11.476146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:11.476146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 616/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:11.480989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:11.580502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66fa6f-e6cc-486b-a923-7a1f7d313cec None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:11.619630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:11.620421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:11.620807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:11.621080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:11.621792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:11.622090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:11.623455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:11.636221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 627/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:11.642550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:11.750274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ae3fe-17ea-449d-a3ba-9a6ebbcbb0da None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:11.786043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:11.787455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:11.787752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:11.788035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:11.795730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:11.797334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:41:11.811141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:11.817172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:11.818262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 617/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:11.818845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:11.925215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f032cb-c101-4081-8ca2-2cf36e36deda None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:11.967029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:11.967780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:11.968062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:11.968338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:11.969164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:11.969726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:11.972431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f277e-beac-48e6-bd17-7d7c8429d823 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:11.986855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 628/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:11.993597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:12.117300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee898-ee55-466e-97c4-0d4b6cd9091a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:12.158856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:12.159572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:12.159894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:12.160167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:12.161081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] RBAC: Adding query filter params (role.id=1dd3964ba91e4f5084105ef2d01f66ce, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=c103464c1bca4da29e61fe73b6fd1d92) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:12.161399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:12.163013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f62512-f5c3-4eb8-b28e-9011c189e396 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:12.169024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 618/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:41:11 2022] GET /identity/v3/role_assignments?user.id=c103464c1bca4da29e61fe73b6fd1d92&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=1dd3964ba91e4f5084105ef2d01f66ce => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:14.497061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f307e6-ee19-4e8c-a8a9-f4b1e01d7677 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:14.497674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f307e6-ee19-4e8c-a8a9-f4b1e01d7677 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:14.497674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f307e6-ee19-4e8c-a8a9-f4b1e01d7677 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:14.498961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 629/1247] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:14.504349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:14.504871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:14.504971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:14.521454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:14.522637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:14.610072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 619/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:14.635042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f307e6-ee19-4e8c-a8a9-f4b1e01d7677 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:14.743484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f307e6-ee19-4e8c-a8a9-f4b1e01d7677 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:14.780707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:14.781756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:14.782033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:14.782302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:14.794002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:41:14.802263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:14.803485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:14.804557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 630/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 16 18:41:14.809925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:14.913286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ba2e0-a2f1-4203-ad27-b7fa9e4f8d8e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:14.948564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:14.949337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:14.949643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:14.949890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:14.950731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:14.951055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:14.952549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:14.962257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 620/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:14.967719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:15.074812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba4edc-8187-47a9-842f-4f2823a58d24 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:15.114374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:15.115175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:15.115542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:15.115800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:15.127077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:15.128565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:15.148440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:15.149598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:15.150673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:15.150673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:15.150673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:15.150673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:15.150673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 631/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:15.155457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:15.259269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58c039-39fe-4243-9f1a-51bcc40d0bfb None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:15.299642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:15.300343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:15.300843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:15.301162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:15.302299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:15.302655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:15.304460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:15.320135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 621/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:15.326438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:15.430634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f98585-175f-4f45-9a9e-d4deaf47fd4c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:15.467673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:15.468703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:15.469110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:15.469421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:15.477699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:15.479122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:41:15.493510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:15.494682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:15.495857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 632/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:15.501277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:15.604172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89430e79-71e3-4ce3-adbf-6e4b96389c3c None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:15.637965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:15.638625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:15.638904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:15.639175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:15.639994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:15.640296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:15.641753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:15.660102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 622/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 16 18:41:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:15.665779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:15.768746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496befee-1a34-4de9-a14b-81bf8594ef89 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:15.803740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:15.804359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:15.804680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:15.804935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:15.870061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:15.871444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:15.968881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 633/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:41:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:41:15.975114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:16.075803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37731a44-2b41-45d2-ae85-01476477352f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:16.119360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:16.120474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:16.120868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:16.121185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:16.133376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:16.134784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:16.152003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:16.153249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:16.154156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:16.154156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:16.154156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:16.154156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:16.154156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 623/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:16.159201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:16.265180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aba67d-9449-49f0-bc32-1a6d5e97321e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:16.301873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:16.302553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:16.302834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:16.303108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:16.303848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:16.304154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:16.305588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:16.317657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 634/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:16.323618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:16.425287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80a897-8e47-48ca-83c2-b308c5eefd1a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:16.460819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:16.461614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:16.461895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:16.462168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:16.469559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:16.471073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:41:16.486507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:16.487658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:16.488875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 624/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:16.493942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:16.593336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a79a-d19c-45e1-bb02-ba42ebd140e6 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:16.629506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:16.629506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:16.629506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:16.629506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:16.630477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:16.630477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:16.633719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266d9c5-d82e-4b62-ac22-729506ff6b4f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:16.643841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 635/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:16.650466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:16.748285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a39bd-c92d-4530-aa94-f8be74ddc9ba None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:16.782082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:16.782731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:16.783013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:16.783295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:16.784112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=c103464c1bca4da29e61fe73b6fd1d92) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:16.784421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:16.785965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e7383-65ca-4c9c-833f-e00f0a84401d None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:16.791478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 625/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:41:16 2022] GET /identity/v3/role_assignments?user.id=c103464c1bca4da29e61fe73b6fd1d92&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:18.953556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b85798-c960-43d2-9df5-60c1c071d9dc None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:18.954114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b85798-c960-43d2-9df5-60c1c071d9dc None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:18.954114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b85798-c960-43d2-9df5-60c1c071d9dc None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:18.955487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 636/1261] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:18.960929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:18.961303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:18.961488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:18.978302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:18.979470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:19.041612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 626/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:19.064124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b85798-c960-43d2-9df5-60c1c071d9dc None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:19.164646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b85798-c960-43d2-9df5-60c1c071d9dc None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:19.196586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011223e9-2290-4865-89ed-506eea736175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:19.197381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011223e9-2290-4865-89ed-506eea736175 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:19.197661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011223e9-2290-4865-89ed-506eea736175 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:19.197928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011223e9-2290-4865-89ed-506eea736175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:19.207438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011223e9-2290-4865-89ed-506eea736175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-011223e9-2290-4865-89ed-506eea736175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:41:19.214922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:19.215991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:19.217203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 637/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:19.222152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:19.320217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d871-0c73-4df7-bdcf-62f86194791e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:19.351733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:19.352379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:19.352676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:19.352949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:19.353674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:19.353973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:19.355314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:19.362753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 627/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:19.368559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011223e9-2290-4865-89ed-506eea736175 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:19.462877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011223e9-2290-4865-89ed-506eea736175 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:19.494321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:19.495076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:19.495353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:19.495620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:19.505491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:19.506911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:19.523506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:19.524532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:19.525771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:19.525771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:19.525771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:19.525771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:19.525771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 638/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:19.530089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:19.622113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42c9ce-cef3-4e02-90ce-04824af32b88 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:19.653510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:19.654142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:19.654417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:19.654683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:19.655378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:19.655676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:19.657281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:19.668146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 628/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:19.673722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:19.767537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691cd85-163e-450d-8fd5-298ab52b05f9 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:19.798894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:19.799661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:19.799935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:19.800199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:19.808834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:19.810257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:41:19.824367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:19.825466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:19.826617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 639/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:19.833908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:19.925750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e47ae7-ab18-4a7c-ba91-42c42f863edf None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:19.956995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:19.958227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:19.958507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:19.958775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:19.960083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:19.960083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:19.961301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:19.977977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 629/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:19.983553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:20.083629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1f1b3-00d8-42b6-b475-3ecf412c42a1 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:20.114819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:20.115589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:20.115867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:20.116138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:20.126030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:20.127472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:20.143706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:20.144779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:20.146430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:20.146430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:20.146430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:20.146430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:20.146430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 640/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:20.151053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:20.243244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b71ac8-7544-4071-8a66-605e57311fa7 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:20.274329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:20.274985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:20.275247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:20.275518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:20.276218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:20.276520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:20.277891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:20.289425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 630/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:20.295045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:20.388334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22bc7f-d6d2-46e0-8931-c3d7b144791d None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:20.420045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:20.420826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:20.421104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:20.421375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:20.427694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:20.429138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:41:20.443282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:20.444296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:20.445272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 641/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:41:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:20.450032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:20.542074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914020e-0935-499e-8a2a-3b98bb840eaa None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:20.572989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:20.573623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:20.573911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:20.574172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:20.574941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:20.575245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:20.576666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09092d-680f-4bb6-bf7a-6227d47fdd65 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:20.588468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 631/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:41:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:20.594253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:20.686561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9504b50-5e7f-4b4a-8063-1a6db900b51e None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:20.719377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27cb90-3e47-4529-b3e5-986f4c016b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:20.720243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27cb90-3e47-4529-b3e5-986f4c016b68 None admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:20.720517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27cb90-3e47-4529-b3e5-986f4c016b68 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:20.720817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27cb90-3e47-4529-b3e5-986f4c016b68 None admin] PATH_INFO: `/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/c103464c1bca4da29e61fe73b6fd1d92/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:20.722712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27cb90-3e47-4529-b3e5-986f4c016b68 None admin] RBAC: Authorizing `identity:create_grant(project_id=25fe1818fd6341f49e3e7ccebcadfc1a, user_id=c103464c1bca4da29e61fe73b6fd1d92, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:20.724110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27cb90-3e47-4529-b3e5-986f4c016b68 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:20.737141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 642/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 16 18:41:20 2022] PUT /identity/v3/projects/25fe1818fd6341f49e3e7ccebcadfc1a/users/c103464c1bca4da29e61fe73b6fd1d92/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:41:22.968883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d396ebf-b3de-4352-9c45-d86d1ea81cfc None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:22.969342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d396ebf-b3de-4352-9c45-d86d1ea81cfc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:22.969447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d396ebf-b3de-4352-9c45-d86d1ea81cfc None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:22.970777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 632/1274] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:22.976740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:22.977124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:22.977295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:22.993901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:22.995057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:23.056920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 643/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:23.083489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d396ebf-b3de-4352-9c45-d86d1ea81cfc None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:23.178225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d396ebf-b3de-4352-9c45-d86d1ea81cfc None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:23.209319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:23.210017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:23.210291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:23.210558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:23.219874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(role_id) May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 16 18:41:23.227902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:23.229080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:23.230281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 633/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:23.234974 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:23.326450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aa157-a0d5-43d2-8ac5-8abdfc4448d5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:23.358788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:23.359446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:23.359724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:23.359995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:23.360756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:23.361060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:23.362414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:23.370302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 644/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:23.376032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:23.470020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65c83-88e7-4afa-a815-22b2e3b41361 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:23.500795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:23.501564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:23.501843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:23.502114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:23.512501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:23.513963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:23.530627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:23.531906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:23.532888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:23.532888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:23.532888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:23.532888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:23.532888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 634/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:23.537558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:23.630456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea36a86-6498-4a7e-9159-bfe694d1b50f None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:23.664266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:23.664945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:23.665224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:23.665495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:23.666202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:23.666506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:23.667860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:23.678995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 645/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:23.684599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:23.779736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7478f39-6445-4c93-9dc8-feb6c01dca2a None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:23.810710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:23.811564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:23.811845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:23.812114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:23.819586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:23.821046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_user(user_id) May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 16 18:41:23.834666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:23.835745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:23.837020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 635/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:23.841406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:23.935926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506cf30-f602-43e7-b742-be4cdb5eb399 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:23.968697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:23.969359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:23.969635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:23.969906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:23.970729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:23.971034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:23.972460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:23.989205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 646/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:23.994858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:24.098780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456088f-d82a-4198-b959-71cfe44b6349 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:24.133190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:24.133806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:24.134083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:24.134350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:24.195945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:24.197390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:24.286976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 636/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 16 18:41:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 16 18:41:24.292833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:24.383834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d728b-fda6-4bec-9fa8-67c34a98238c None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:24.415929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:24.416712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:24.417063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:24.417339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:24.427104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:24.428523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 16 18:41:24.444937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:24.445948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:24.447266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 16 18:41:24.447266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 16 18:41:24.447266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 16 18:41:24.447266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:24.447266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 647/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:24.451487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:24.542621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540c54c-6fe2-4653-b345-bbf25bcf6850 None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:24.572898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:24.573527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:24.573805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:24.574074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:24.574769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:24.575068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:24.576396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:24.587227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 637/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 16 18:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:24.592457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:24.684693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b16a-ebc7-42ab-8ed6-6cf69d8457bf None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:24.715973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:24.716752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:24.717029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:24.717298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:24.724102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:24.725529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(project_id) May 16 18:41:24.738540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application with Lock( May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:24.739569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: ERROR keystone.server.flask.application May 16 18:41:24.740849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 648/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 16 18:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:24.745403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:24.836940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f8c63-f5b6-4b7c-8326-dc7799a916ae None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:24.867485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:24.868109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:24.868385 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:24.868675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:24.869454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:24.869757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:24.871145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609775f7-6cc5-48cd-81bb-71eda07817c2 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:24.883558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 638/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 16 18:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:41:24.889401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:24.982375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774cc1f-defb-416a-bbfe-9f3ff5b39fce None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:25.013240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:25.013873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:25.014148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:25.014416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:25.015224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] RBAC: Adding query filter params (role.id=461cbc289f54486bb7c5bb877fa01923, scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a, user.id=c103464c1bca4da29e61fe73b6fd1d92) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:25.015526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:25.016956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45004d-7b37-4598-96cf-3291d6c0f223 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:25.023187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 649/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 16 18:41:24 2022] GET /identity/v3/role_assignments?user.id=c103464c1bca4da29e61fe73b6fd1d92&scope.project.id=25fe1818fd6341f49e3e7ccebcadfc1a&role.id=461cbc289f54486bb7c5bb877fa01923 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:27.267235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb0aa3-29ae-41f4-b1df-37397af8d4c7 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:27.267747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb0aa3-29ae-41f4-b1df-37397af8d4c7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:27.267826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb0aa3-29ae-41f4-b1df-37397af8d4c7 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:27.269503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 639/1288] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:27.276191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbdf44-d100-4403-966f-747049afceeb None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:27.276609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbdf44-d100-4403-966f-747049afceeb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:27.276926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbdf44-d100-4403-966f-747049afceeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:27.296989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-50fbdf44-d100-4403-966f-747049afceeb None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:27.298432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-50fbdf44-d100-4403-966f-747049afceeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:27.364420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 650/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:27.386332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb0aa3-29ae-41f4-b1df-37397af8d4c7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:27.481810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb0aa3-29ae-41f4-b1df-37397af8d4c7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:27.513178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:27.513951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:27.514228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:27.514495 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] PATH_INFO: `/v3/services/placement` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:27.515132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:27.516477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:27.528431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:27.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:27.530892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 640/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:41:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:27.536352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbdf44-d100-4403-966f-747049afceeb None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:27.627501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbdf44-d100-4403-966f-747049afceeb None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:27.659111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:27.659750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:27.660029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:27.660885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:27.661586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:27.661889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:27.663243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5f706-6f9c-4a92-b08f-f15bc0c017c7 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:27.667791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 651/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 16 18:41:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:27.673388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:27.772259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3164c-753c-4e71-8606-4980bc86edff None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:27.803900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:27.804474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:27.804782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:27.805056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] PATH_INFO: `/v3/services` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:27.805743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:27.806045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:27.807387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712b6e0-27a1-4517-aaa1-c10d2527d9a4 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:27.811520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 641/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 16 18:41:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:29.977534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78260dcc-2570-4411-b4aa-a93eac2e7983 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:29.977930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78260dcc-2570-4411-b4aa-a93eac2e7983 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:29.978080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78260dcc-2570-4411-b4aa-a93eac2e7983 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:29.979433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 652/1293] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:29.984430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b432cd-dd4e-4042-8ebc-5513e025e2a2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:29.984902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b432cd-dd4e-4042-8ebc-5513e025e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:29.984995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b432cd-dd4e-4042-8ebc-5513e025e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:30.001118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d7b432cd-dd4e-4042-8ebc-5513e025e2a2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:30.002291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d7b432cd-dd4e-4042-8ebc-5513e025e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:30.065153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 642/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:30.088860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78260dcc-2570-4411-b4aa-a93eac2e7983 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:30.196582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78260dcc-2570-4411-b4aa-a93eac2e7983 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:30.227927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca136b3-e416-4f39-98a4-406c627b3c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:30.228504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca136b3-e416-4f39-98a4-406c627b3c69 None admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:30.228812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca136b3-e416-4f39-98a4-406c627b3c69 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:30.229086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca136b3-e416-4f39-98a4-406c627b3c69 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:30.229784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca136b3-e416-4f39-98a4-406c627b3c69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:30.231131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca136b3-e416-4f39-98a4-406c627b3c69 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:30.243709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 653/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 16 18:41:30 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:41:32.436358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5b29a-330e-4bdf-b177-9bdcbbcf855c None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:32.436924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5b29a-330e-4bdf-b177-9bdcbbcf855c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:32.436924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5b29a-330e-4bdf-b177-9bdcbbcf855c None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:32.438270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 643/1296] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:32.443463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:32.443955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:32.444051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:32.460793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:32.461957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:32.531405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 654/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:32.554946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5b29a-330e-4bdf-b177-9bdcbbcf855c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:32.646778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5b29a-330e-4bdf-b177-9bdcbbcf855c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:32.678968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:32.679736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:32.680016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:32.680283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] PATH_INFO: `/v3/services/placement` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:32.680948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:32.682296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:32.693638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:32.694810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:32.695940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 644/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:41:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 140 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:32.700494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:32.791557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c6fb4-7a15-4915-865d-de47327acf06 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:32.822115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:32.822747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:32.823024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:32.823291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:32.823978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:32.824277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:32.825644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:32.830405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 655/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 16 18:41:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:32.836444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:32.929766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918c2d-4ea5-47d9-900d-d7d7515ffd1f None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:32.961297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:32.961938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:32.962221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:32.962496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:32.963304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] RBAC: Adding query filter params (interface=public, service_id=5f06c9f9c3834784bf27db3404362cdf, region_id=RegionOne) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:32.963612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:32.964996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427edf9-09e0-4d1b-ac2b-1d1d25833120 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:32.969834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 645/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 16 18:41:32 2022] GET /identity/v3/endpoints?service_id=5f06c9f9c3834784bf27db3404362cdf&interface=public®ion_id=RegionOne => generated 194 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:32.975928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:33.073776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4323f-2b2a-4d99-8e48-acc5cf481f72 None admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:33.109914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074d39e-23eb-4603-9f1b-a243cc48acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:33.110619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074d39e-23eb-4603-9f1b-a243cc48acd5 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:33.110874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074d39e-23eb-4603-9f1b-a243cc48acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:33.111143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074d39e-23eb-4603-9f1b-a243cc48acd5 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:33.111813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074d39e-23eb-4603-9f1b-a243cc48acd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:33.113220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074d39e-23eb-4603-9f1b-a243cc48acd5 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:33.120823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 656/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 16 18:41:32 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:41:35.328117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62a119-f7d9-458e-94c0-0a5143f78985 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:35.328719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62a119-f7d9-458e-94c0-0a5143f78985 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:35.328719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62a119-f7d9-458e-94c0-0a5143f78985 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:35.330030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 646/1302] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:41:35.335276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:35.335648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:35.335826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:35.351769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:41:35.352944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:41:35.415526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 657/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:41:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 16 18:41:35.438168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62a119-f7d9-458e-94c0-0a5143f78985 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:35.531810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62a119-f7d9-458e-94c0-0a5143f78985 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:35.563040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:35.563800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:35.564076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:35.564344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:35.564999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:35.566344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application Traceback (most recent call last): May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 16 18:41:35.577512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._get_service(service_id) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 16 18:41:35.578731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.get_or_create( May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application with Lock( May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self._enter() May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application return self.creator() May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application created_value = creator( May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: ERROR keystone.server.flask.application May 16 18:41:35.579949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 647/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 16 18:41:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 16 18:41:35.584552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:35.677344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a9c9b-6d80-4bbe-bb03-b373eaddbe61 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:35.708298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:35.708968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:35.709247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:35.709516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] PATH_INFO: `/v3/services` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:35.710213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:41:35.710514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:35.711860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573b0c8-1695-4714-bcc6-40d6542a4af2 None admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:35.716765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 658/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 16 18:41:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:41:35.722913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:41:35.824452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbd304-bad2-48b2-a794-0ea6eed2b9bd None admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:41:35.855701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da2194-db27-4a0e-bd77-4758a26c6709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:41:35.856276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da2194-db27-4a0e-bd77-4758a26c6709 None admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:41:35.856552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da2194-db27-4a0e-bd77-4758a26c6709 None admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:41:35.856847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da2194-db27-4a0e-bd77-4758a26c6709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:41:35.857560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da2194-db27-4a0e-bd77-4758a26c6709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:41:35.858904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da2194-db27-4a0e-bd77-4758a26c6709 None admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:41:35.875662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 648/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 16 18:41:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:42:39.448533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c4ff3-992a-4772-89d1-f130de4eb633 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:39.449611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c4ff3-992a-4772-89d1-f130de4eb633 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:39.450100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c4ff3-992a-4772-89d1-f130de4eb633 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:39.451676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 659/1307] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:39.456954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc6c83-9e0f-41e6-a95b-e54d287373a0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:39.457805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc6c83-9e0f-41e6-a95b-e54d287373a0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:39.458310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc6c83-9e0f-41e6-a95b-e54d287373a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:39.555313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b1fc6c83-9e0f-41e6-a95b-e54d287373a0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:39.557099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b1fc6c83-9e0f-41e6-a95b-e54d287373a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:39.693283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 649/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:39.745715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488cd9d-ac91-43ef-be16-d6287c4c17d8 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:39.746342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488cd9d-ac91-43ef-be16-d6287c4c17d8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:39.746948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488cd9d-ac91-43ef-be16-d6287c4c17d8 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:39.749977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 660/1309] 10.222.0.36 () {54 vars in 964 bytes} [Mon May 16 18:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:39.759533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a031-6bc1-442e-85d9-3bffb41e13e8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:39.759758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a031-6bc1-442e-85d9-3bffb41e13e8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:39.760098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a031-6bc1-442e-85d9-3bffb41e13e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:39.916182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-23d3a031-6bc1-442e-85d9-3bffb41e13e8 None None] MFA Rules not processed for user `e653231605f345b0846ac42508e7d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:39.917838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-23d3a031-6bc1-442e-85d9-3bffb41e13e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:40.006423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 650/1310] 10.222.0.36 () {58 vars in 1061 bytes} [Mon May 16 18:42:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:40.015583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488cd9d-ac91-43ef-be16-d6287c4c17d8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:40.017376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f488cd9d-ac91-43ef-be16-d6287c4c17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:40.130026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488cd9d-ac91-43ef-be16-d6287c4c17d8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:40.201632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:40.202326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:40.202614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:40.202891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:40.204533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:40.313227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:40.314978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dd6e6-b1dc-439e-80b5-85234ecb6648 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:40.335861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 661/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:42:42.844392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834030e7-d32a-4eab-b7c0-047803fa7413 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:42.844936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834030e7-d32a-4eab-b7c0-047803fa7413 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:42.845032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834030e7-d32a-4eab-b7c0-047803fa7413 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:42.846413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 651/1312] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:42.851899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14c28-7e7c-4b51-8ae9-d721c636422d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:42.852334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14c28-7e7c-4b51-8ae9-d721c636422d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:42.852454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14c28-7e7c-4b51-8ae9-d721c636422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:42.935890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b4d14c28-7e7c-4b51-8ae9-d721c636422d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:42.937125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b4d14c28-7e7c-4b51-8ae9-d721c636422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:43.037557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 662/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:43.096184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f54db-fed9-4d55-b720-4ad2ea2258c3 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:43.096763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f54db-fed9-4d55-b720-4ad2ea2258c3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:43.097112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f54db-fed9-4d55-b720-4ad2ea2258c3 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:43.098591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 652/1314] 10.222.0.36 () {54 vars in 964 bytes} [Mon May 16 18:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:43.104698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055ac1b-10da-463d-879f-f5cce7d52460 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:43.104993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055ac1b-10da-463d-879f-f5cce7d52460 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:43.105262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055ac1b-10da-463d-879f-f5cce7d52460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:43.180400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b055ac1b-10da-463d-879f-f5cce7d52460 None None] MFA Rules not processed for user `e653231605f345b0846ac42508e7d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:43.181521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b055ac1b-10da-463d-879f-f5cce7d52460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:43.256289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 663/1315] 10.222.0.36 () {58 vars in 1061 bytes} [Mon May 16 18:42:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:43.265193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f54db-fed9-4d55-b720-4ad2ea2258c3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:43.267391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-721f54db-fed9-4d55-b720-4ad2ea2258c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:43.369092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f54db-fed9-4d55-b720-4ad2ea2258c3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:43.440108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:43.441021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:43.441364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:43.441702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:43.443409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:43.552614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:43.554306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5337f-19d6-4704-8093-1f501fdea234 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:43.572591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 653/1316] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:42:46.366195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc211-c9ac-4ab3-ac58-4a9bd1683206 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:46.366803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc211-c9ac-4ab3-ac58-4a9bd1683206 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:46.366803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc211-c9ac-4ab3-ac58-4a9bd1683206 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:46.368053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 664/1317] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:46.373091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bfdcd-0b4e-477c-bc41-58a86c7a5509 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:46.373499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bfdcd-0b4e-477c-bc41-58a86c7a5509 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:46.374551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bfdcd-0b4e-477c-bc41-58a86c7a5509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:46.434695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-359bfdcd-0b4e-477c-bc41-58a86c7a5509 None None] MFA Rules not processed for user `a908d4f350ba4702b4193b2939c5a7be`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:46.435885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-359bfdcd-0b4e-477c-bc41-58a86c7a5509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:46.525510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 654/1318] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:46.564782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efc211-c9ac-4ab3-ac58-4a9bd1683206 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:46.566287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-85efc211-c9ac-4ab3-ac58-4a9bd1683206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:46.662652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efc211-c9ac-4ab3-ac58-4a9bd1683206 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:46.729621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:46.730247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:46.730523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:46.730792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:46.732370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:46.835066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:46.836487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd0936-ce14-4f15-95ed-847fa4d827b9 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:46.853832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 665/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:42:49.567882 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7f8f9-f053-493c-865a-50859569fa4d None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:49.568435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7f8f9-f053-493c-865a-50859569fa4d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:49.568435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7f8f9-f053-493c-865a-50859569fa4d None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:49.569813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 655/1320] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:49.575015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6774-0370-496d-8929-810a65c5f2e1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:49.575440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6774-0370-496d-8929-810a65c5f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:49.575558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6774-0370-496d-8929-810a65c5f2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:49.638393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-116b6774-0370-496d-8929-810a65c5f2e1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:49.639581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-116b6774-0370-496d-8929-810a65c5f2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:49.733823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 666/1321] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:49.772277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7f8f9-f053-493c-865a-50859569fa4d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:49.773843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-84c7f8f9-f053-493c-865a-50859569fa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:49.870874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7f8f9-f053-493c-865a-50859569fa4d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:49.940915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:49.941553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:49.941833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:49.942102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:49.943741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:50.045685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:50.047679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e05e1-9af5-47c7-95e6-925a51c72ab2 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:50.065074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 656/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:42:52.940582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7fabc-ad6a-486e-83e6-8f7d21176ad5 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:52.941174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7fabc-ad6a-486e-83e6-8f7d21176ad5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:52.941174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7fabc-ad6a-486e-83e6-8f7d21176ad5 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:52.942517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 667/1323] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:52.947755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75f9fc-ad9e-491b-9be5-5176cad4d59a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:52.948196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75f9fc-ad9e-491b-9be5-5176cad4d59a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:52.948320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75f9fc-ad9e-491b-9be5-5176cad4d59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:53.010541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4e75f9fc-ad9e-491b-9be5-5176cad4d59a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:53.011717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4e75f9fc-ad9e-491b-9be5-5176cad4d59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:53.103383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 657/1324] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:53.139334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7fabc-ad6a-486e-83e6-8f7d21176ad5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:53.140978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0ee7fabc-ad6a-486e-83e6-8f7d21176ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:53.242108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7fabc-ad6a-486e-83e6-8f7d21176ad5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:53.312576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:53.313239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:53.314011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:53.314288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:53.315953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:53.420490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:53.421976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65c2af-0b39-4409-9f11-d04a0776b44e service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:53.443733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 668/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:42:55.735992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576695d4-c2ac-41dc-903b-dd0e0f078bdb None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:55.736497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576695d4-c2ac-41dc-903b-dd0e0f078bdb None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:55.736497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576695d4-c2ac-41dc-903b-dd0e0f078bdb None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:55.737843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 658/1326] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:55.742964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995cd2d-0813-4d9d-a3ef-730a94ff9399 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:55.743408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995cd2d-0813-4d9d-a3ef-730a94ff9399 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:55.743505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995cd2d-0813-4d9d-a3ef-730a94ff9399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:55.806287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1995cd2d-0813-4d9d-a3ef-730a94ff9399 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:55.807711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1995cd2d-0813-4d9d-a3ef-730a94ff9399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:55.898984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 669/1327] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:55.934563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576695d4-c2ac-41dc-903b-dd0e0f078bdb None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:55.936091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-576695d4-c2ac-41dc-903b-dd0e0f078bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:56.035414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576695d4-c2ac-41dc-903b-dd0e0f078bdb None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:56.106065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:56.106704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:56.106984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:56.107261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:56.108924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:56.212255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:56.213757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b1f9f-8a55-469c-8b22-673b00129dd4 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:56.234260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 659/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:42:58.505077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079f9f5-de85-44a1-8162-af926ccd95a2 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:58.505641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079f9f5-de85-44a1-8162-af926ccd95a2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:58.505641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079f9f5-de85-44a1-8162-af926ccd95a2 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:58.506994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 670/1329] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:42:58.512326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ce6a-a179-4593-983d-e35a2e644502 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:58.512781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ce6a-a179-4593-983d-e35a2e644502 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:58.512927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ce6a-a179-4593-983d-e35a2e644502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:58.576428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4411ce6a-a179-4593-983d-e35a2e644502 None None] MFA Rules not processed for user `a908d4f350ba4702b4193b2939c5a7be`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:42:58.577637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4411ce6a-a179-4593-983d-e35a2e644502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:58.668150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 660/1330] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:42:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:42:58.704858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079f9f5-de85-44a1-8162-af926ccd95a2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:42:58.706358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c079f9f5-de85-44a1-8162-af926ccd95a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:58.807225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079f9f5-de85-44a1-8162-af926ccd95a2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:42:58.888061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:42:58.888740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:42:58.889020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:42:58.889289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:42:58.890911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:42:58.995582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:42:58.997056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a803ea9-f71e-488c-9cbe-e637fe27b312 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:42:59.014993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 671/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:42:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:01.651194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f9ee-f415-4604-9586-570452026806 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:01.651738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f9ee-f415-4604-9586-570452026806 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:01.651738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f9ee-f415-4604-9586-570452026806 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:01.653258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 661/1332] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:01.659402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaef13-fd52-4ef0-b3a6-b727671eac76 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:01.659862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaef13-fd52-4ef0-b3a6-b727671eac76 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:01.659958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaef13-fd52-4ef0-b3a6-b727671eac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:01.725757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-12aaef13-fd52-4ef0-b3a6-b727671eac76 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:01.726954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-12aaef13-fd52-4ef0-b3a6-b727671eac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:01.827825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 672/1333] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:01.863731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f9ee-f415-4604-9586-570452026806 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:01.865267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0007f9ee-f415-4604-9586-570452026806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:01.964322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f9ee-f415-4604-9586-570452026806 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:02.037281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:02.037928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:02.038207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:02.038475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:02.040124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:02.152448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:02.153920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c259-df24-4c1d-96e0-abdec85fb848 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:02.175826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 662/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:04.437146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5339df-52de-4ca1-8327-ee547b2351ad None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:04.437722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5339df-52de-4ca1-8327-ee547b2351ad None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:04.437722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5339df-52de-4ca1-8327-ee547b2351ad None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:04.439537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 673/1335] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:04.444747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf17c4-6350-4c53-b7a6-80ebe40c00e5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:04.445171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf17c4-6350-4c53-b7a6-80ebe40c00e5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:04.445299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf17c4-6350-4c53-b7a6-80ebe40c00e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:04.506663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d9cf17c4-6350-4c53-b7a6-80ebe40c00e5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:04.507839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d9cf17c4-6350-4c53-b7a6-80ebe40c00e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:04.602806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 663/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:04.642227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5339df-52de-4ca1-8327-ee547b2351ad None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:04.643747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5e5339df-52de-4ca1-8327-ee547b2351ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:04.742384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5339df-52de-4ca1-8327-ee547b2351ad None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:04.810648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:04.811277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:04.811583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:04.811849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:04.813507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:04.925927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:04.927347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e789d9-9e9f-4e01-9165-c5bc6c1b566b service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:04.944762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 674/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:07.458164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd20ba-079c-4871-a58e-40480fb30f9f None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:07.458621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd20ba-079c-4871-a58e-40480fb30f9f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:07.458721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd20ba-079c-4871-a58e-40480fb30f9f None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:07.460068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 664/1338] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:43:07.465349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d392-ae30-41c8-9b15-36b3640224bc None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:07.465806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d392-ae30-41c8-9b15-36b3640224bc None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:07.465903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d392-ae30-41c8-9b15-36b3640224bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:07.527465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1d55d392-ae30-41c8-9b15-36b3640224bc None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:07.528661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1d55d392-ae30-41c8-9b15-36b3640224bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:07.621568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 675/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:07.656623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd20ba-079c-4871-a58e-40480fb30f9f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:07.658152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-30dd20ba-079c-4871-a58e-40480fb30f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:07.754776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd20ba-079c-4871-a58e-40480fb30f9f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:07.822715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:07.823363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:07.823639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:07.823910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:07.825553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:07.929113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:07.930536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16644a1-a419-4182-94d4-74fdf646a00b service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:07.948113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 665/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:12.108794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf285d64-8acd-4319-aa83-63c2e92109b6 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:12.109314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf285d64-8acd-4319-aa83-63c2e92109b6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:12.109314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf285d64-8acd-4319-aa83-63c2e92109b6 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:12.110692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 676/1341] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:12.116091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328beefd-094b-4a1d-8daf-f309ba194096 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:12.116505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328beefd-094b-4a1d-8daf-f309ba194096 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:12.116722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328beefd-094b-4a1d-8daf-f309ba194096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:12.186135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-328beefd-094b-4a1d-8daf-f309ba194096 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:12.187362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-328beefd-094b-4a1d-8daf-f309ba194096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:12.288133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 666/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:12.327419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf285d64-8acd-4319-aa83-63c2e92109b6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:12.329573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cf285d64-8acd-4319-aa83-63c2e92109b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:12.420683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf285d64-8acd-4319-aa83-63c2e92109b6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:12.425006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c529c7-4897-4b8e-9f2d-468d0b554ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:12.425644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c529c7-4897-4b8e-9f2d-468d0b554ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:12.425923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c529c7-4897-4b8e-9f2d-468d0b554ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:12.426195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c529c7-4897-4b8e-9f2d-468d0b554ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:12.434994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c529c7-4897-4b8e-9f2d-468d0b554ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:12.436414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c529c7-4897-4b8e-9f2d-468d0b554ad1 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:12.446327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 677/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:15.276138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18a3a-2c03-49f3-be68-aad365b48dd3 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:15.276705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18a3a-2c03-49f3-be68-aad365b48dd3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:15.276705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18a3a-2c03-49f3-be68-aad365b48dd3 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:15.278062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 667/1344] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:15.283370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866d192-96e3-4328-bc25-847510d347c9 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:15.283812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866d192-96e3-4328-bc25-847510d347c9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:15.283914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866d192-96e3-4328-bc25-847510d347c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:15.316323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c866d192-96e3-4328-bc25-847510d347c9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:15.317536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c866d192-96e3-4328-bc25-847510d347c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:15.389820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 678/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:15.426177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18a3a-2c03-49f3-be68-aad365b48dd3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:15.436443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18a3a-2c03-49f3-be68-aad365b48dd3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:15.440651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17dc5b-3a40-4d68-8ce5-e70f0a12be03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:15.441288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17dc5b-3a40-4d68-8ce5-e70f0a12be03 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:15.441563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17dc5b-3a40-4d68-8ce5-e70f0a12be03 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:15.441827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17dc5b-3a40-4d68-8ce5-e70f0a12be03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:15.450327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17dc5b-3a40-4d68-8ce5-e70f0a12be03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:15.451731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17dc5b-3a40-4d68-8ce5-e70f0a12be03 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:15.461268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 668/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:19.593791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bef50-529c-4607-a83d-87a493277786 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:19.594355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bef50-529c-4607-a83d-87a493277786 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:19.594355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bef50-529c-4607-a83d-87a493277786 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:19.595698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 679/1347] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:19.600919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468563-d0d7-4c5c-ab15-69d482f7a33b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:19.601380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468563-d0d7-4c5c-ab15-69d482f7a33b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:19.601483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468563-d0d7-4c5c-ab15-69d482f7a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:19.618799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b9468563-d0d7-4c5c-ab15-69d482f7a33b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:19.619971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b9468563-d0d7-4c5c-ab15-69d482f7a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:19.629017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 669/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:43:19.670104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bef50-529c-4607-a83d-87a493277786 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:19.680697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bef50-529c-4607-a83d-87a493277786 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:19.684360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4af161-843e-4730-8faa-87da25220d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:19.685026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4af161-843e-4730-8faa-87da25220d44 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:19.685306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4af161-843e-4730-8faa-87da25220d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:19.685579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4af161-843e-4730-8faa-87da25220d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:19.694187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4af161-843e-4730-8faa-87da25220d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:19.695598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4af161-843e-4730-8faa-87da25220d44 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:19.705227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 680/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:25.103779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2a2d-46f7-4bf5-9f1b-531fa6138d82 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:25.104697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2a2d-46f7-4bf5-9f1b-531fa6138d82 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:25.105184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2a2d-46f7-4bf5-9f1b-531fa6138d82 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:25.107190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 670/1350] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:25.113898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4911d-fc6c-4d6f-9642-69a170d8ad06 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:25.115241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4911d-fc6c-4d6f-9642-69a170d8ad06 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:25.115720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4911d-fc6c-4d6f-9642-69a170d8ad06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:25.194610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-71a4911d-fc6c-4d6f-9642-69a170d8ad06 None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:25.197296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-71a4911d-fc6c-4d6f-9642-69a170d8ad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:25.274590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 681/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:43:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:29.480298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaca3f-d975-4c63-826f-063b0b584c87 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:29.480819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaca3f-d975-4c63-826f-063b0b584c87 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:29.480905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaca3f-d975-4c63-826f-063b0b584c87 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:29.482377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 671/1352] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:29.488022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ec6a8-9ae5-4053-85ed-8044fe956fc5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:29.488983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ec6a8-9ae5-4053-85ed-8044fe956fc5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:29.489469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ec6a8-9ae5-4053-85ed-8044fe956fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:29.510915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8b1ec6a8-9ae5-4053-85ed-8044fe956fc5 None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:29.512744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b1ec6a8-9ae5-4053-85ed-8044fe956fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:29.522653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 682/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:43:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:31.386620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69479dae-cb79-488a-83b4-7e9807517c07 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:31.387150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69479dae-cb79-488a-83b4-7e9807517c07 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:31.387241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69479dae-cb79-488a-83b4-7e9807517c07 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:31.388522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 672/1354] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:31.394102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b46e3-159c-4b07-8520-ce4be3a66d74 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:31.394529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b46e3-159c-4b07-8520-ce4be3a66d74 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:31.394653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b46e3-159c-4b07-8520-ce4be3a66d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:31.412518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-e98b46e3-159c-4b07-8520-ce4be3a66d74 None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:31.413780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e98b46e3-159c-4b07-8520-ce4be3a66d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:31.422582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 683/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:43:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:43:35.204614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a37f9-4256-4790-b981-2afb76b0c7af None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:35.205161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a37f9-4256-4790-b981-2afb76b0c7af None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:35.205161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a37f9-4256-4790-b981-2afb76b0c7af None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:35.206600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 673/1356] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:35.211963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58563d49-81f4-4bda-b52f-cc1c22dd5bee None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:35.212390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58563d49-81f4-4bda-b52f-cc1c22dd5bee None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:35.212498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58563d49-81f4-4bda-b52f-cc1c22dd5bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:35.229542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-58563d49-81f4-4bda-b52f-cc1c22dd5bee None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:35.230694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-58563d49-81f4-4bda-b52f-cc1c22dd5bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:35.238603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 684/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:43:35.286007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6bd6-b844-4eb5-b805-4aa3c44624a9 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:35.286291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6bd6-b844-4eb5-b805-4aa3c44624a9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:35.286564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc6bd6-b844-4eb5-b805-4aa3c44624a9 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:35.288033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 674/1358] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 16 18:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:35.293220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244086-a3a3-4295-a065-3c78f631f564 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:35.293507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244086-a3a3-4295-a065-3c78f631f564 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:35.293787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244086-a3a3-4295-a065-3c78f631f564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:35.341435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7b244086-a3a3-4295-a065-3c78f631f564 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:35.342636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7b244086-a3a3-4295-a065-3c78f631f564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:35.421076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 685/1359] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 16 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:43:35.429654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc6bd6-b844-4eb5-b805-4aa3c44624a9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:35.444696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc6bd6-b844-4eb5-b805-4aa3c44624a9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:35.450103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8ccf4-1356-4a66-ab14-b2d53cb74c28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:35.450728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ccf4-1356-4a66-ab14-b2d53cb74c28 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:35.451009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ccf4-1356-4a66-ab14-b2d53cb74c28 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:35.451280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ccf4-1356-4a66-ab14-b2d53cb74c28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:35.459857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8ccf4-1356-4a66-ab14-b2d53cb74c28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:35.461511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8ccf4-1356-4a66-ab14-b2d53cb74c28 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:35.473485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 675/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:37.944979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edebce-6cff-4ab3-92f2-0008a6d61950 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:37.945545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edebce-6cff-4ab3-92f2-0008a6d61950 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:37.945545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edebce-6cff-4ab3-92f2-0008a6d61950 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:37.947059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 686/1361] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:37.952880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead7854-6a2f-4ce5-8664-9fb2221e6c41 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:37.953395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead7854-6a2f-4ce5-8664-9fb2221e6c41 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:37.953493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead7854-6a2f-4ce5-8664-9fb2221e6c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:37.975542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4ead7854-6a2f-4ce5-8664-9fb2221e6c41 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:37.977199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4ead7854-6a2f-4ce5-8664-9fb2221e6c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:38.000405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 676/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:38.044205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edebce-6cff-4ab3-92f2-0008a6d61950 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:38.062870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edebce-6cff-4ab3-92f2-0008a6d61950 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:38.069666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c4915-7014-4ba9-a3e7-1d9b7578d598 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:38.070492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c4915-7014-4ba9-a3e7-1d9b7578d598 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:38.070993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c4915-7014-4ba9-a3e7-1d9b7578d598 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:38.071301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c4915-7014-4ba9-a3e7-1d9b7578d598 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:38.082739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c4915-7014-4ba9-a3e7-1d9b7578d598 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:38.084408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c4915-7014-4ba9-a3e7-1d9b7578d598 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:38.098182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 687/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:40.618142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8cb7a-e447-4a38-8580-71ab6729eb5e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:40.618648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8cb7a-e447-4a38-8580-71ab6729eb5e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:40.618743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8cb7a-e447-4a38-8580-71ab6729eb5e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:40.620029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 677/1364] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:40.625260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba510c-b3b9-4942-ae79-359aac3b966b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:40.625650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba510c-b3b9-4942-ae79-359aac3b966b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:40.625870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba510c-b3b9-4942-ae79-359aac3b966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:40.644106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-06ba510c-b3b9-4942-ae79-359aac3b966b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:40.645321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-06ba510c-b3b9-4942-ae79-359aac3b966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:40.653308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 688/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:43:40.698036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc983f-4e9b-49a0-adbc-758d192129ac None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:40.698321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc983f-4e9b-49a0-adbc-758d192129ac None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:40.698599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc983f-4e9b-49a0-adbc-758d192129ac None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:40.699958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 678/1366] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 16 18:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:40.705936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef07153-d7e2-433b-8ff4-aa30f1fea2f7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:40.706321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef07153-d7e2-433b-8ff4-aa30f1fea2f7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:40.706668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef07153-d7e2-433b-8ff4-aa30f1fea2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:40.725184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-4ef07153-d7e2-433b-8ff4-aa30f1fea2f7 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:40.726618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4ef07153-d7e2-433b-8ff4-aa30f1fea2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:40.736872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 689/1367] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 16 18:43:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:40.744698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc983f-4e9b-49a0-adbc-758d192129ac None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:40.761088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc983f-4e9b-49a0-adbc-758d192129ac None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:40.767070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24eea56-3e6f-4d93-bba8-06cd156b42e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:40.767698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eea56-3e6f-4d93-bba8-06cd156b42e7 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:40.767978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eea56-3e6f-4d93-bba8-06cd156b42e7 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:40.768254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eea56-3e6f-4d93-bba8-06cd156b42e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:40.781888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24eea56-3e6f-4d93-bba8-06cd156b42e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:40.783590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24eea56-3e6f-4d93-bba8-06cd156b42e7 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:40.810327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 679/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:43:43.098379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9691a6-ec39-4eb3-aab4-719a971e98ec None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.098929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9691a6-ec39-4eb3-aab4-719a971e98ec None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.098929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9691a6-ec39-4eb3-aab4-719a971e98ec None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.100853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 690/1369] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:43.106216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dc9f4-b1f7-43f4-8c83-483a21a7e20b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.106652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dc9f4-b1f7-43f4-8c83-483a21a7e20b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.106764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dc9f4-b1f7-43f4-8c83-483a21a7e20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.129057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-211dc9f4-b1f7-43f4-8c83-483a21a7e20b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:43.130264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-211dc9f4-b1f7-43f4-8c83-483a21a7e20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:43.138499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 680/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:43.183031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9691a6-ec39-4eb3-aab4-719a971e98ec None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:43.198935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9691a6-ec39-4eb3-aab4-719a971e98ec None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:43.204834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b21fec-ad35-436b-959c-2830d8f3fdd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:43.205563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b21fec-ad35-436b-959c-2830d8f3fdd4 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.205911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b21fec-ad35-436b-959c-2830d8f3fdd4 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.206248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b21fec-ad35-436b-959c-2830d8f3fdd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.215602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b21fec-ad35-436b-959c-2830d8f3fdd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:43.218776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b21fec-ad35-436b-959c-2830d8f3fdd4 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:43.229764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 691/1371] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:43.513629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d7b5c-1760-49ab-ad41-3766c39a2df0 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.514060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d7b5c-1760-49ab-ad41-3766c39a2df0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.514313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d7b5c-1760-49ab-ad41-3766c39a2df0 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.515672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 681/1372] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:43.521052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d233-0ed4-4fb5-b8e5-aa4c1e6f0b70 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.521533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d233-0ed4-4fb5-b8e5-aa4c1e6f0b70 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.521814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d233-0ed4-4fb5-b8e5-aa4c1e6f0b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.541115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c764d233-0ed4-4fb5-b8e5-aa4c1e6f0b70 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:43.542303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c764d233-0ed4-4fb5-b8e5-aa4c1e6f0b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:43.550888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 692/1373] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:43.842865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f415e0-3c37-4730-9ea2-b09ae13fb840 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.843340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f415e0-3c37-4730-9ea2-b09ae13fb840 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.843684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f415e0-3c37-4730-9ea2-b09ae13fb840 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.845159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 682/1374] 10.222.0.36 () {54 vars in 963 bytes} [Mon May 16 18:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:43.850325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7190-7962-44d4-91fe-84bc7b78f8b2 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.850609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7190-7962-44d4-91fe-84bc7b78f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.850883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7190-7962-44d4-91fe-84bc7b78f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:43.900880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-113a7190-7962-44d4-91fe-84bc7b78f8b2 None None] MFA Rules not processed for user `dfc10bc677cc4cf4b673fdcbd937d352`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:43.902202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-113a7190-7962-44d4-91fe-84bc7b78f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:43.969310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 693/1375] 10.222.0.36 () {58 vars in 1060 bytes} [Mon May 16 18:43:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:43.976877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f415e0-3c37-4730-9ea2-b09ae13fb840 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:43.987922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f415e0-3c37-4730-9ea2-b09ae13fb840 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:43.991705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc231-c716-47a0-bdcc-74ecab9b32d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:43.992416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc231-c716-47a0-bdcc-74ecab9b32d6 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:43.992851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc231-c716-47a0-bdcc-74ecab9b32d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:43.993137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc231-c716-47a0-bdcc-74ecab9b32d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:44.003050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc231-c716-47a0-bdcc-74ecab9b32d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:44.004555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc231-c716-47a0-bdcc-74ecab9b32d6 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:44.014512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 683/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:43:45.492606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86021693-b268-4275-9f32-f4495c562a81 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:45.493188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86021693-b268-4275-9f32-f4495c562a81 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:45.493188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86021693-b268-4275-9f32-f4495c562a81 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:45.494524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 694/1377] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:45.499806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4b6e-ed55-435e-aa14-c59120c6d9e1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:45.500283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4b6e-ed55-435e-aa14-c59120c6d9e1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:45.500477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4b6e-ed55-435e-aa14-c59120c6d9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:45.518853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-8cbd4b6e-ed55-435e-aa14-c59120c6d9e1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:45.520078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8cbd4b6e-ed55-435e-aa14-c59120c6d9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:45.528024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 684/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:43:45.573620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86021693-b268-4275-9f32-f4495c562a81 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:45.588330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86021693-b268-4275-9f32-f4495c562a81 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:45.594542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591bfde-4acc-4d5b-abe3-352fb10c4e33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:45.595201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591bfde-4acc-4d5b-abe3-352fb10c4e33 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:45.595483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591bfde-4acc-4d5b-abe3-352fb10c4e33 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:45.595757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591bfde-4acc-4d5b-abe3-352fb10c4e33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:45.604851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591bfde-4acc-4d5b-abe3-352fb10c4e33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:45.606614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591bfde-4acc-4d5b-abe3-352fb10c4e33 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:45.617944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 695/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:47.912448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb661-5db7-4a05-b841-84e8303e3521 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:47.913059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb661-5db7-4a05-b841-84e8303e3521 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:47.913059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb661-5db7-4a05-b841-84e8303e3521 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:47.914368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 685/1380] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:47.919574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e82c1-bc8b-40ae-9d5e-47689f73cb17 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:47.920017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e82c1-bc8b-40ae-9d5e-47689f73cb17 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:47.920163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e82c1-bc8b-40ae-9d5e-47689f73cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:47.938175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-987e82c1-bc8b-40ae-9d5e-47689f73cb17 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:47.939591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-987e82c1-bc8b-40ae-9d5e-47689f73cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:47.947826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 696/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:47.991921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb661-5db7-4a05-b841-84e8303e3521 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:48.007259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb661-5db7-4a05-b841-84e8303e3521 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:48.012718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e30ade-d9dd-4101-9881-3633dcbf8ea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:48.013360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e30ade-d9dd-4101-9881-3633dcbf8ea5 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:48.014311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e30ade-d9dd-4101-9881-3633dcbf8ea5 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:48.014591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e30ade-d9dd-4101-9881-3633dcbf8ea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:48.023270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e30ade-d9dd-4101-9881-3633dcbf8ea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:48.024873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e30ade-d9dd-4101-9881-3633dcbf8ea5 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:48.035996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 686/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:50.295571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd505798-2d53-4788-af8e-1caf6d3ed5b2 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:50.296155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd505798-2d53-4788-af8e-1caf6d3ed5b2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:50.296155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd505798-2d53-4788-af8e-1caf6d3ed5b2 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:50.297521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 697/1383] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:50.302961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c02b-e002-404f-8289-c2830fea9d8f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:50.303435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c02b-e002-404f-8289-c2830fea9d8f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:50.303530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c02b-e002-404f-8289-c2830fea9d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:50.321469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5840c02b-e002-404f-8289-c2830fea9d8f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:50.322673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5840c02b-e002-404f-8289-c2830fea9d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:50.332017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 687/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:50.374152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd505798-2d53-4788-af8e-1caf6d3ed5b2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:50.388752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd505798-2d53-4788-af8e-1caf6d3ed5b2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:50.394279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986458b-9294-4765-adc0-4dc518218f3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:50.395050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986458b-9294-4765-adc0-4dc518218f3a service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:50.395431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986458b-9294-4765-adc0-4dc518218f3a service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:50.395807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986458b-9294-4765-adc0-4dc518218f3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:50.404596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986458b-9294-4765-adc0-4dc518218f3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:50.406416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986458b-9294-4765-adc0-4dc518218f3a service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:50.416104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 698/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:52.665323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303bc2-67e7-4219-b0d6-d190b3830abd None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:52.666050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303bc2-67e7-4219-b0d6-d190b3830abd None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:52.666050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303bc2-67e7-4219-b0d6-d190b3830abd None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:52.667217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 688/1386] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:52.673661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342f08-172e-4a56-9854-aa079c66b592 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:52.674096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342f08-172e-4a56-9854-aa079c66b592 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:52.674415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342f08-172e-4a56-9854-aa079c66b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:52.697463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-72342f08-172e-4a56-9854-aa079c66b592 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:52.699091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-72342f08-172e-4a56-9854-aa079c66b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:52.709563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 699/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:52.757684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303bc2-67e7-4219-b0d6-d190b3830abd None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:52.775381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303bc2-67e7-4219-b0d6-d190b3830abd None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:52.781530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15e046-5d3a-4e86-b935-44d6c127a278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:52.782163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e046-5d3a-4e86-b935-44d6c127a278 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:52.782439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e046-5d3a-4e86-b935-44d6c127a278 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:52.782708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e046-5d3a-4e86-b935-44d6c127a278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:52.791374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e046-5d3a-4e86-b935-44d6c127a278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:52.792973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e046-5d3a-4e86-b935-44d6c127a278 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:52.802551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 689/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:55.170169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73dcd5-6ad7-4374-9cae-1d37caeb160a None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:55.170796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73dcd5-6ad7-4374-9cae-1d37caeb160a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:55.171009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73dcd5-6ad7-4374-9cae-1d37caeb160a None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:55.173151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 700/1389] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:55.178573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d476861-cbd5-4040-9d3d-685c091e88e3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:55.179119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d476861-cbd5-4040-9d3d-685c091e88e3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:55.179119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d476861-cbd5-4040-9d3d-685c091e88e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:55.201050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6d476861-cbd5-4040-9d3d-685c091e88e3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:55.203834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6d476861-cbd5-4040-9d3d-685c091e88e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:55.213948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 690/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:55.258906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73dcd5-6ad7-4374-9cae-1d37caeb160a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:55.274343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73dcd5-6ad7-4374-9cae-1d37caeb160a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:55.280251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7a15a-ea6e-4d84-abf4-96eabe6c1126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:55.280903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7a15a-ea6e-4d84-abf4-96eabe6c1126 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:55.281181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7a15a-ea6e-4d84-abf4-96eabe6c1126 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:55.281452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7a15a-ea6e-4d84-abf4-96eabe6c1126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:55.289320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7a15a-ea6e-4d84-abf4-96eabe6c1126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:55.290918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7a15a-ea6e-4d84-abf4-96eabe6c1126 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:55.300025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 701/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:58.011743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ad40-253c-4693-a627-ec5346eb8946 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:58.012330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ad40-253c-4693-a627-ec5346eb8946 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:58.012330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ad40-253c-4693-a627-ec5346eb8946 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:58.013677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 691/1392] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:58.019014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78f37c-0a3c-43c0-b183-5dc2ba09bf49 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:58.019461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78f37c-0a3c-43c0-b183-5dc2ba09bf49 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:58.019624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78f37c-0a3c-43c0-b183-5dc2ba09bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:58.037436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-cb78f37c-0a3c-43c0-b183-5dc2ba09bf49 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:58.038647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cb78f37c-0a3c-43c0-b183-5dc2ba09bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:58.046922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 702/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:58.092098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ad40-253c-4693-a627-ec5346eb8946 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:43:58.106110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ad40-253c-4693-a627-ec5346eb8946 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:43:58.111944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327bd04-ff27-4720-a2a5-933905cce305 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:43:58.112566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bd04-ff27-4720-a2a5-933905cce305 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:58.112884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bd04-ff27-4720-a2a5-933905cce305 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:58.113151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327bd04-ff27-4720-a2a5-933905cce305 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:58.121066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327bd04-ff27-4720-a2a5-933905cce305 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:43:58.122638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327bd04-ff27-4720-a2a5-933905cce305 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:43:58.132437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 692/1394] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:43:59.792103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0060-3bb8-48aa-a5a3-aef525f09469 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:59.792693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0060-3bb8-48aa-a5a3-aef525f09469 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:59.792693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a0060-3bb8-48aa-a5a3-aef525f09469 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:59.794093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 703/1395] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:59.799921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b794d-f582-482f-b0c7-63949d7ce49c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:59.800356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b794d-f582-482f-b0c7-63949d7ce49c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:59.800615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b794d-f582-482f-b0c7-63949d7ce49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:59.817729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a63b794d-f582-482f-b0c7-63949d7ce49c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:59.818902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a63b794d-f582-482f-b0c7-63949d7ce49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:59.826750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 693/1396] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:43:59.881460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfff28f-2259-45b5-86e5-c63d0847ff17 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:59.881744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfff28f-2259-45b5-86e5-c63d0847ff17 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:59.882019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfff28f-2259-45b5-86e5-c63d0847ff17 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:59.883359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 704/1397] 10.222.0.36 () {54 vars in 962 bytes} [Mon May 16 18:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:43:59.888213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0901f-51a6-4db4-a6fa-9051024e872c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:43:59.888500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0901f-51a6-4db4-a6fa-9051024e872c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:43:59.888818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0901f-51a6-4db4-a6fa-9051024e872c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:43:59.937979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0fd0901f-51a6-4db4-a6fa-9051024e872c None None] MFA Rules not processed for user `fbdba20bd1c243248348d5c5672fefbb`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:43:59.939307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0fd0901f-51a6-4db4-a6fa-9051024e872c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:43:59.997019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 694/1398] 10.222.0.36 () {58 vars in 1059 bytes} [Mon May 16 18:43:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:00.004307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfff28f-2259-45b5-86e5-c63d0847ff17 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:00.014533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfff28f-2259-45b5-86e5-c63d0847ff17 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:00.018038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e0eaa-be56-47ee-9edb-d47dd23d03f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:00.018658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0eaa-be56-47ee-9edb-d47dd23d03f9 service glance] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.018936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0eaa-be56-47ee-9edb-d47dd23d03f9 service glance] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.019223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0eaa-be56-47ee-9edb-d47dd23d03f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.028269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e0eaa-be56-47ee-9edb-d47dd23d03f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:00.030014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e0eaa-be56-47ee-9edb-d47dd23d03f9 service glance] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:00.039747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 705/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:00.366410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7efc95-4697-4773-bd28-bb1c9baba9ec None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.366696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7efc95-4697-4773-bd28-bb1c9baba9ec None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.366973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7efc95-4697-4773-bd28-bb1c9baba9ec None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.368330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 695/1400] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:00.373723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f43e-a9bc-49f5-af08-805446b5a2a3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.373948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f43e-a9bc-49f5-af08-805446b5a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.374261 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f43e-a9bc-49f5-af08-805446b5a2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.394510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-17d9f43e-a9bc-49f5-af08-805446b5a2a3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:00.395709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-17d9f43e-a9bc-49f5-af08-805446b5a2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:00.404012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 706/1401] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:00.449301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7efc95-4697-4773-bd28-bb1c9baba9ec None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:00.465631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7efc95-4697-4773-bd28-bb1c9baba9ec None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:00.466681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df803ce-dfc0-4495-9dcd-dbb303275c04 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.466965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df803ce-dfc0-4495-9dcd-dbb303275c04 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.467240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df803ce-dfc0-4495-9dcd-dbb303275c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.471401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1e304-1e6f-4d69-a7b2-b0331247128c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:00.472062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e304-1e6f-4d69-a7b2-b0331247128c service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.472346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e304-1e6f-4d69-a7b2-b0331247128c service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.472620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e304-1e6f-4d69-a7b2-b0331247128c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.481075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1e304-1e6f-4d69-a7b2-b0331247128c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:00.482652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1e304-1e6f-4d69-a7b2-b0331247128c service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:00.491969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 696/1402] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:00.515670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-2df803ce-dfc0-4495-9dcd-dbb303275c04 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:00.516875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2df803ce-dfc0-4495-9dcd-dbb303275c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:00.581580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 707/1403] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:44:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:44:00.625359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d035a-37eb-4a8f-800e-3f7eab636415 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.625571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d035a-37eb-4a8f-800e-3f7eab636415 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.625832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d035a-37eb-4a8f-800e-3f7eab636415 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.627290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 697/1404] 10.222.0.36 () {54 vars in 942 bytes} [Mon May 16 18:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:00.634223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85731135-a33e-4294-b64f-a5643c5db201 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.634619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85731135-a33e-4294-b64f-a5643c5db201 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.634956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85731135-a33e-4294-b64f-a5643c5db201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.688073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-85731135-a33e-4294-b64f-a5643c5db201 None None] MFA Rules not processed for user `16b2497bffb3487bbd025f22463d5a79`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:00.689327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-85731135-a33e-4294-b64f-a5643c5db201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:00.762353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 708/1405] 10.222.0.36 () {58 vars in 1039 bytes} [Mon May 16 18:44:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:00.772401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d035a-37eb-4a8f-800e-3f7eab636415 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:00.785818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d035a-37eb-4a8f-800e-3f7eab636415 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:00.790753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb685651-4f36-4444-aaf3-8ebff5fd767e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:00.791381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb685651-4f36-4444-aaf3-8ebff5fd767e service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:00.791665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb685651-4f36-4444-aaf3-8ebff5fd767e service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:00.791941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb685651-4f36-4444-aaf3-8ebff5fd767e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:00.799846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb685651-4f36-4444-aaf3-8ebff5fd767e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:00.801488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb685651-4f36-4444-aaf3-8ebff5fd767e service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:00.809427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 698/1406] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:44:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:44:02.777645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677569f-fe75-4a02-b6bf-953458c8908d None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:02.778185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677569f-fe75-4a02-b6bf-953458c8908d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:02.778185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677569f-fe75-4a02-b6bf-953458c8908d None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:02.779830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 709/1407] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:02.785819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db489121-1eb9-49c2-b2c2-89e01712f7c3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:02.786256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db489121-1eb9-49c2-b2c2-89e01712f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:02.786355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db489121-1eb9-49c2-b2c2-89e01712f7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:02.805009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-db489121-1eb9-49c2-b2c2-89e01712f7c3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:02.806213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-db489121-1eb9-49c2-b2c2-89e01712f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:02.813965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 699/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:44:02.857490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677569f-fe75-4a02-b6bf-953458c8908d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:02.871782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677569f-fe75-4a02-b6bf-953458c8908d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:02.879496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1ee83-64ba-4d6c-a286-c020a00b60d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:02.880137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1ee83-64ba-4d6c-a286-c020a00b60d3 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:02.880412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1ee83-64ba-4d6c-a286-c020a00b60d3 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:02.880714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1ee83-64ba-4d6c-a286-c020a00b60d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:02.889332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1ee83-64ba-4d6c-a286-c020a00b60d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:02.890972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1ee83-64ba-4d6c-a286-c020a00b60d3 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:02.900592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 710/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:24.319664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba305-87e9-43af-b3af-51724b57c161 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:24.320316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba305-87e9-43af-b3af-51724b57c161 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:24.320316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba305-87e9-43af-b3af-51724b57c161 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:24.321573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 700/1410] 10.222.0.36 () {54 vars in 918 bytes} [Mon May 16 18:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:24.326620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87d1e-cea4-4845-acc7-abdc4eb1912b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:24.327052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87d1e-cea4-4845-acc7-abdc4eb1912b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:24.327204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87d1e-cea4-4845-acc7-abdc4eb1912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:24.345807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-93a87d1e-cea4-4845-acc7-abdc4eb1912b None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:24.346969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-93a87d1e-cea4-4845-acc7-abdc4eb1912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:24.355173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 711/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Mon May 16 18:44:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:24.368899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd919c3-71c6-4b8a-9f34-0c6f3d485b31 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:24.369185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd919c3-71c6-4b8a-9f34-0c6f3d485b31 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:24.369456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd919c3-71c6-4b8a-9f34-0c6f3d485b31 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:24.370757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 701/1412] 10.222.0.36 () {54 vars in 918 bytes} [Mon May 16 18:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:24.375007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8c24-c313-4f6a-80e1-104c138a9b22 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:24.375290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8c24-c313-4f6a-80e1-104c138a9b22 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:24.375563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8c24-c313-4f6a-80e1-104c138a9b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:24.394056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-198d8c24-c313-4f6a-80e1-104c138a9b22 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:24.395199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-198d8c24-c313-4f6a-80e1-104c138a9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:24.404028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 712/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Mon May 16 18:44:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:27.048070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8085-a426-475d-9e63-3caf834e8f4e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:27.048684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8085-a426-475d-9e63-3caf834e8f4e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:27.048684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8085-a426-475d-9e63-3caf834e8f4e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:27.050038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:27.055298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781676e-0c00-4283-97c0-aa3274e62733 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:27.055697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781676e-0c00-4283-97c0-aa3274e62733 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:27.055803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781676e-0c00-4283-97c0-aa3274e62733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:27.073371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8781676e-0c00-4283-97c0-aa3274e62733 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:27.074515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8781676e-0c00-4283-97c0-aa3274e62733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:27.082042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:27.122441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c8085-a426-475d-9e63-3caf834e8f4e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:27.136962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c8085-a426-475d-9e63-3caf834e8f4e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:27.142311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865608a6-439d-49b9-ba73-6926a6373115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:27.142926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865608a6-439d-49b9-ba73-6926a6373115 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:27.143207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865608a6-439d-49b9-ba73-6926a6373115 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:27.143479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865608a6-439d-49b9-ba73-6926a6373115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:27.151208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865608a6-439d-49b9-ba73-6926a6373115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:27.152822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865608a6-439d-49b9-ba73-6926a6373115 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:27.162243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 703/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:28.144933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e686c-49d6-4581-9e1b-04954fc53c9b None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:28.145516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e686c-49d6-4581-9e1b-04954fc53c9b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:28.145516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e686c-49d6-4581-9e1b-04954fc53c9b None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:28.146843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:28.151683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1772c-ec8f-4795-b2c0-5051032873d2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:28.152157 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1772c-ec8f-4795-b2c0-5051032873d2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:28.152265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1772c-ec8f-4795-b2c0-5051032873d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:28.170803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f4a1772c-ec8f-4795-b2c0-5051032873d2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:28.172019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f4a1772c-ec8f-4795-b2c0-5051032873d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:28.180240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:28.346606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e686c-49d6-4581-9e1b-04954fc53c9b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:28.360347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e686c-49d6-4581-9e1b-04954fc53c9b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:28.365625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2500a-8dcd-4141-9c7b-0abd3f282aa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:28.366277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2500a-8dcd-4141-9c7b-0abd3f282aa4 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:28.366563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2500a-8dcd-4141-9c7b-0abd3f282aa4 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:28.366832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2500a-8dcd-4141-9c7b-0abd3f282aa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:28.374549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2500a-8dcd-4141-9c7b-0abd3f282aa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:28.376602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2500a-8dcd-4141-9c7b-0abd3f282aa4 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:28.385604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 715/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:29.325668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56ad92-9fb8-4407-87b0-f102507dec35 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:29.326209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56ad92-9fb8-4407-87b0-f102507dec35 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:29.326209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56ad92-9fb8-4407-87b0-f102507dec35 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:29.327549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:29.332546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ecc14-6509-4f24-8cd4-85579e943323 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:29.332982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ecc14-6509-4f24-8cd4-85579e943323 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:29.333133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ecc14-6509-4f24-8cd4-85579e943323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:29.350425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ff2ecc14-6509-4f24-8cd4-85579e943323 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:29.351592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ff2ecc14-6509-4f24-8cd4-85579e943323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:29.359179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:29.516187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56ad92-9fb8-4407-87b0-f102507dec35 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:29.530000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56ad92-9fb8-4407-87b0-f102507dec35 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:29.535484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9826e9c-c29d-45d2-8e41-acc9784a9b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:29.536105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9826e9c-c29d-45d2-8e41-acc9784a9b8d service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:29.536379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9826e9c-c29d-45d2-8e41-acc9784a9b8d service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:29.536678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9826e9c-c29d-45d2-8e41-acc9784a9b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:29.544339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9826e9c-c29d-45d2-8e41-acc9784a9b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:29.545920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9826e9c-c29d-45d2-8e41-acc9784a9b8d service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:29.555286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 706/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:30.466962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0162e63-934e-48d2-bfb1-6b4738711a80 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:30.467539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0162e63-934e-48d2-bfb1-6b4738711a80 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:30.467539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0162e63-934e-48d2-bfb1-6b4738711a80 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:30.469491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:30.474615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2415-848f-4ff5-adc8-55bde3890e46 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:30.475020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2415-848f-4ff5-adc8-55bde3890e46 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:30.475197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2415-848f-4ff5-adc8-55bde3890e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:30.493021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f55d2415-848f-4ff5-adc8-55bde3890e46 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:30.494186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f55d2415-848f-4ff5-adc8-55bde3890e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:30.502071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:30.648283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0162e63-934e-48d2-bfb1-6b4738711a80 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:30.662716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0162e63-934e-48d2-bfb1-6b4738711a80 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:30.668210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41875b7-0905-40c6-ad3d-cca7357ed55e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:30.668865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41875b7-0905-40c6-ad3d-cca7357ed55e service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:30.669143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41875b7-0905-40c6-ad3d-cca7357ed55e service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:30.669412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41875b7-0905-40c6-ad3d-cca7357ed55e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:30.677885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41875b7-0905-40c6-ad3d-cca7357ed55e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:30.679436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41875b7-0905-40c6-ad3d-cca7357ed55e service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:30.688381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 718/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:31.614331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56949-3399-451e-b250-c607f1bd0cc9 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:31.614891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56949-3399-451e-b250-c607f1bd0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:31.614891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56949-3399-451e-b250-c607f1bd0cc9 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:31.616224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:31.621081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a852-2c42-454d-86b7-e6374554f254 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:31.621468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a852-2c42-454d-86b7-e6374554f254 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:31.621667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5a852-2c42-454d-86b7-e6374554f254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:31.639679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-bba5a852-2c42-454d-86b7-e6374554f254 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:31.640892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bba5a852-2c42-454d-86b7-e6374554f254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:31.651198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:31.800219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f56949-3399-451e-b250-c607f1bd0cc9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:31.814885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f56949-3399-451e-b250-c607f1bd0cc9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:31.820191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298ddb0-d2b9-4908-8bc7-b0cdb759ee69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:31.820846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298ddb0-d2b9-4908-8bc7-b0cdb759ee69 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:31.821128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298ddb0-d2b9-4908-8bc7-b0cdb759ee69 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:31.821403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298ddb0-d2b9-4908-8bc7-b0cdb759ee69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:31.829247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298ddb0-d2b9-4908-8bc7-b0cdb759ee69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:31.830815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298ddb0-d2b9-4908-8bc7-b0cdb759ee69 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:31.839818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 709/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:32.712612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61beeed-04a0-4a71-8fc6-a0b1fd05d339 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:32.713171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61beeed-04a0-4a71-8fc6-a0b1fd05d339 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:32.713306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61beeed-04a0-4a71-8fc6-a0b1fd05d339 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:32.714567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:32.719443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c871aa8-2b91-42d9-b874-5d4297ac2402 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:32.719829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c871aa8-2b91-42d9-b874-5d4297ac2402 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:32.719988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c871aa8-2b91-42d9-b874-5d4297ac2402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:32.737121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4c871aa8-2b91-42d9-b874-5d4297ac2402 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:32.738285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4c871aa8-2b91-42d9-b874-5d4297ac2402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:32.745709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:32.897820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61beeed-04a0-4a71-8fc6-a0b1fd05d339 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:32.912266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61beeed-04a0-4a71-8fc6-a0b1fd05d339 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:32.917637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaebc7-51ec-4eca-a44e-181f33a4f1ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:32.918267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaebc7-51ec-4eca-a44e-181f33a4f1ca service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:32.918542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaebc7-51ec-4eca-a44e-181f33a4f1ca service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:32.918812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaebc7-51ec-4eca-a44e-181f33a4f1ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:32.926781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaebc7-51ec-4eca-a44e-181f33a4f1ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:32.928337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaebc7-51ec-4eca-a44e-181f33a4f1ca service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:32.937357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 721/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:33.829035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c005d77-214d-408f-a754-13132560ea7b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:33.829634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c005d77-214d-408f-a754-13132560ea7b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:33.829634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c005d77-214d-408f-a754-13132560ea7b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:33.830957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:33.835949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1850-86c7-4234-a8d8-cdfa783983e8 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:33.836402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1850-86c7-4234-a8d8-cdfa783983e8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:33.836514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1850-86c7-4234-a8d8-cdfa783983e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:33.854256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-393b1850-86c7-4234-a8d8-cdfa783983e8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:33.855437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-393b1850-86c7-4234-a8d8-cdfa783983e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:33.863750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:34.013729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c005d77-214d-408f-a754-13132560ea7b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:34.028112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c005d77-214d-408f-a754-13132560ea7b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:34.034342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759d9df-705b-48f1-b628-ec406dba6c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:34.034966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759d9df-705b-48f1-b628-ec406dba6c5c service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:34.035248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759d9df-705b-48f1-b628-ec406dba6c5c service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:34.035521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759d9df-705b-48f1-b628-ec406dba6c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:34.043354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759d9df-705b-48f1-b628-ec406dba6c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:34.044958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759d9df-705b-48f1-b628-ec406dba6c5c service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:34.054035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 712/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:34.915756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1530389-7e66-4a78-b763-eeb65050e2c2 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:34.916294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1530389-7e66-4a78-b763-eeb65050e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:34.916294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1530389-7e66-4a78-b763-eeb65050e2c2 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:34.917745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:34.922530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c928fc-5658-4c0c-878a-5afc9a7fd77f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:34.922945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c928fc-5658-4c0c-878a-5afc9a7fd77f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:34.923073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c928fc-5658-4c0c-878a-5afc9a7fd77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:34.940665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-03c928fc-5658-4c0c-878a-5afc9a7fd77f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:34.941825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-03c928fc-5658-4c0c-878a-5afc9a7fd77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:34.949375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:35.098135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1530389-7e66-4a78-b763-eeb65050e2c2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:35.112884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1530389-7e66-4a78-b763-eeb65050e2c2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:35.118287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84b713-2542-4002-b9b5-00d1500a74b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:35.118911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84b713-2542-4002-b9b5-00d1500a74b6 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:35.119192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84b713-2542-4002-b9b5-00d1500a74b6 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:35.119462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84b713-2542-4002-b9b5-00d1500a74b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:35.127376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84b713-2542-4002-b9b5-00d1500a74b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:35.128992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84b713-2542-4002-b9b5-00d1500a74b6 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:35.138190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 724/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:35.994955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bed1f-0b5e-46b9-8cfe-2f4affd4f5e3 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:35.995420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bed1f-0b5e-46b9-8cfe-2f4affd4f5e3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:35.995513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bed1f-0b5e-46b9-8cfe-2f4affd4f5e3 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:35.996866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:36.001787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4c283-030c-45f2-8090-9271e9ecf4b0 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:36.002200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4c283-030c-45f2-8090-9271e9ecf4b0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:36.002337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4c283-030c-45f2-8090-9271e9ecf4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:36.020418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c7d4c283-030c-45f2-8090-9271e9ecf4b0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:36.021641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c7d4c283-030c-45f2-8090-9271e9ecf4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:36.029503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:36.177861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bed1f-0b5e-46b9-8cfe-2f4affd4f5e3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:36.191797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bed1f-0b5e-46b9-8cfe-2f4affd4f5e3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:36.197222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785ff4e-360a-43c9-8e6b-3494dc7b6295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:36.197847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ff4e-360a-43c9-8e6b-3494dc7b6295 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:36.198134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ff4e-360a-43c9-8e6b-3494dc7b6295 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:36.198431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ff4e-360a-43c9-8e6b-3494dc7b6295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:36.206737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785ff4e-360a-43c9-8e6b-3494dc7b6295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:36.208302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785ff4e-360a-43c9-8e6b-3494dc7b6295 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:36.217505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 715/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:38.310594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e15e-e818-4aea-b657-6dc131c1dd92 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:38.311161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e15e-e818-4aea-b657-6dc131c1dd92 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:38.311161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e15e-e818-4aea-b657-6dc131c1dd92 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:38.312520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:38.318042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a8a5-aaf8-437a-9475-44797e0add89 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:38.318475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a8a5-aaf8-437a-9475-44797e0add89 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:38.318583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a8a5-aaf8-437a-9475-44797e0add89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:38.335909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f910a8a5-aaf8-437a-9475-44797e0add89 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:38.337115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f910a8a5-aaf8-437a-9475-44797e0add89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:38.345253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:44:38.382302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636e15e-e818-4aea-b657-6dc131c1dd92 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:38.393214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636e15e-e818-4aea-b657-6dc131c1dd92 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:38.396729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd467ae6-49dc-44a8-ad77-7dccdbf96f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:38.397355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd467ae6-49dc-44a8-ad77-7dccdbf96f7d service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:38.397637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd467ae6-49dc-44a8-ad77-7dccdbf96f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:38.397908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd467ae6-49dc-44a8-ad77-7dccdbf96f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:38.407300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd467ae6-49dc-44a8-ad77-7dccdbf96f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:38.408733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd467ae6-49dc-44a8-ad77-7dccdbf96f7d service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:38.417777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 727/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:40.775067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fc62e-1acb-4d0a-b848-96ef75d50aae None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:40.775664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fc62e-1acb-4d0a-b848-96ef75d50aae None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:40.775664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fc62e-1acb-4d0a-b848-96ef75d50aae None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:40.777005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:40.783155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468170f-bc30-4bf0-9895-fd009ae74374 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:40.783598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468170f-bc30-4bf0-9895-fd009ae74374 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:40.783731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468170f-bc30-4bf0-9895-fd009ae74374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:40.800780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-3468170f-bc30-4bf0-9895-fd009ae74374 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:40.801945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3468170f-bc30-4bf0-9895-fd009ae74374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:40.809560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:44:40.847523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fc62e-1acb-4d0a-b848-96ef75d50aae None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:40.857892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fc62e-1acb-4d0a-b848-96ef75d50aae None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:40.861921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f9eb-f33d-44de-9636-848020d9b409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:40.862546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f9eb-f33d-44de-9636-848020d9b409 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:40.862824 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f9eb-f33d-44de-9636-848020d9b409 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:40.863096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f9eb-f33d-44de-9636-848020d9b409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:40.871677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4f9eb-f33d-44de-9636-848020d9b409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:40.873139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4f9eb-f33d-44de-9636-848020d9b409 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:40.882834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 718/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:43.389204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98911ff-7ed2-4b4d-b904-eac40a50692b None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:43.389770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98911ff-7ed2-4b4d-b904-eac40a50692b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:43.389770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98911ff-7ed2-4b4d-b904-eac40a50692b None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:43.391654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:43.396842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9d58-a656-4780-b048-f6d88be8e8ea None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:43.397271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9d58-a656-4780-b048-f6d88be8e8ea None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:43.397427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9d58-a656-4780-b048-f6d88be8e8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:43.414480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-bf7f9d58-a656-4780-b048-f6d88be8e8ea None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:43.415638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-bf7f9d58-a656-4780-b048-f6d88be8e8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:43.423170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:44:43.461054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98911ff-7ed2-4b4d-b904-eac40a50692b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:43.471723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98911ff-7ed2-4b4d-b904-eac40a50692b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:43.475227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8174-260e-4b02-9433-37789dac2991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:43.475850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8174-260e-4b02-9433-37789dac2991 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:43.476129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8174-260e-4b02-9433-37789dac2991 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:43.476404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8174-260e-4b02-9433-37789dac2991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:43.485187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da8174-260e-4b02-9433-37789dac2991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:43.486598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da8174-260e-4b02-9433-37789dac2991 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:43.495978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 730/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:45.819798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b326-5fe4-4bdf-a65e-bc058bdbcdae None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:45.820391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b326-5fe4-4bdf-a65e-bc058bdbcdae None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:45.820391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b326-5fe4-4bdf-a65e-bc058bdbcdae None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:45.821739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:45.826957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61542-0114-45c4-988c-23c50c2cd84e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:45.827409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61542-0114-45c4-988c-23c50c2cd84e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:45.827518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61542-0114-45c4-988c-23c50c2cd84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:45.844572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-40c61542-0114-45c4-988c-23c50c2cd84e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:45.845755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-40c61542-0114-45c4-988c-23c50c2cd84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:45.853335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:45.891775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b326-5fe4-4bdf-a65e-bc058bdbcdae None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:45.893472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-97d5b326-5fe4-4bdf-a65e-bc058bdbcdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:45.905856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b326-5fe4-4bdf-a65e-bc058bdbcdae None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:45.909712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee0c93-5a15-4e96-90c2-f1650ae3ddfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:45.910339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee0c93-5a15-4e96-90c2-f1650ae3ddfe service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:45.910619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee0c93-5a15-4e96-90c2-f1650ae3ddfe service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:45.910893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee0c93-5a15-4e96-90c2-f1650ae3ddfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:45.919565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee0c93-5a15-4e96-90c2-f1650ae3ddfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:45.921016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee0c93-5a15-4e96-90c2-f1650ae3ddfe service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:45.931093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 721/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:48.294966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d90f3-e273-4c39-af07-93b8625fc9ef None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:48.295530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d90f3-e273-4c39-af07-93b8625fc9ef None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:48.295530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d90f3-e273-4c39-af07-93b8625fc9ef None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:48.296907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:48.302699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69665c5f-23b1-45ad-a506-fa3c7fb02538 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:48.303398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69665c5f-23b1-45ad-a506-fa3c7fb02538 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:48.303622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69665c5f-23b1-45ad-a506-fa3c7fb02538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:48.321967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-69665c5f-23b1-45ad-a506-fa3c7fb02538 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:48.323142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-69665c5f-23b1-45ad-a506-fa3c7fb02538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:48.330760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:48.367196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d90f3-e273-4c39-af07-93b8625fc9ef None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:48.378159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d90f3-e273-4c39-af07-93b8625fc9ef None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:48.381636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897be95-2432-4b49-b74a-0047d4866047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:48.382256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897be95-2432-4b49-b74a-0047d4866047 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:48.382539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897be95-2432-4b49-b74a-0047d4866047 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:48.382813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897be95-2432-4b49-b74a-0047d4866047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:48.391458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897be95-2432-4b49-b74a-0047d4866047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:48.392967 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897be95-2432-4b49-b74a-0047d4866047 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:48.402042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 733/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:50.764461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae793-f61c-4272-884a-7edf8c6d9bb0 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:50.765081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae793-f61c-4272-884a-7edf8c6d9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:50.765081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fae793-f61c-4272-884a-7edf8c6d9bb0 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:50.767020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:50.772276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801ece-c3b3-445a-a005-ebae9a051faf None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:50.772722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801ece-c3b3-445a-a005-ebae9a051faf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:50.772867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801ece-c3b3-445a-a005-ebae9a051faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:50.790137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1d801ece-c3b3-445a-a005-ebae9a051faf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:50.791292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1d801ece-c3b3-445a-a005-ebae9a051faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:50.799076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:50.836354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fae793-f61c-4272-884a-7edf8c6d9bb0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:50.846836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fae793-f61c-4272-884a-7edf8c6d9bb0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:50.851973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d05ec8-b627-4b81-a1e9-0959cb98ffa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:50.852557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d05ec8-b627-4b81-a1e9-0959cb98ffa9 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:50.852866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d05ec8-b627-4b81-a1e9-0959cb98ffa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:50.853135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d05ec8-b627-4b81-a1e9-0959cb98ffa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:50.862080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d05ec8-b627-4b81-a1e9-0959cb98ffa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:50.863485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d05ec8-b627-4b81-a1e9-0959cb98ffa9 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:50.872399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 724/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:53.162208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b35dc-9812-4083-a322-41ac9afed47a None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:53.162777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b35dc-9812-4083-a322-41ac9afed47a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:53.162777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b35dc-9812-4083-a322-41ac9afed47a None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:53.164129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:53.169346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4545a-4ebb-4cfc-be8b-54d7dea15e45 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:53.169767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4545a-4ebb-4cfc-be8b-54d7dea15e45 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:53.169898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4545a-4ebb-4cfc-be8b-54d7dea15e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:53.187006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-79f4545a-4ebb-4cfc-be8b-54d7dea15e45 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:53.188701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-79f4545a-4ebb-4cfc-be8b-54d7dea15e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:53.196230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:53.232667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b35dc-9812-4083-a322-41ac9afed47a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:53.242942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b35dc-9812-4083-a322-41ac9afed47a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:53.246388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24ae98-b2b5-424a-85ae-d6947e8ac95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:53.247001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24ae98-b2b5-424a-85ae-d6947e8ac95b service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:53.247278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24ae98-b2b5-424a-85ae-d6947e8ac95b service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:53.247547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24ae98-b2b5-424a-85ae-d6947e8ac95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:53.256538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24ae98-b2b5-424a-85ae-d6947e8ac95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:53.258083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24ae98-b2b5-424a-85ae-d6947e8ac95b service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:53.267665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 736/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:55.553488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80888379-ffed-4df8-8569-b3017a099423 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:55.553936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80888379-ffed-4df8-8569-b3017a099423 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:55.554043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80888379-ffed-4df8-8569-b3017a099423 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:55.555392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:55.560800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bdc78-7407-46d1-aecc-b047878c3971 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:55.561182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bdc78-7407-46d1-aecc-b047878c3971 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:55.561346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bdc78-7407-46d1-aecc-b047878c3971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:55.578917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ba2bdc78-7407-46d1-aecc-b047878c3971 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:55.580089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ba2bdc78-7407-46d1-aecc-b047878c3971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:55.587646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:55.624311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80888379-ffed-4df8-8569-b3017a099423 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:55.634970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80888379-ffed-4df8-8569-b3017a099423 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:55.638366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f47f47-9fdf-4207-a634-88f2086161b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:55.638978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f47f47-9fdf-4207-a634-88f2086161b7 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:55.639251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f47f47-9fdf-4207-a634-88f2086161b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:55.639518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f47f47-9fdf-4207-a634-88f2086161b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:55.647862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f47f47-9fdf-4207-a634-88f2086161b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:55.649302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f47f47-9fdf-4207-a634-88f2086161b7 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:55.658405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 727/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:44:57.891228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8678-d92b-4879-9988-d666128f9615 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:57.891769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8678-d92b-4879-9988-d666128f9615 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:57.891769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8678-d92b-4879-9988-d666128f9615 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:57.893158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:44:57.898488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bdb5c-33be-47ef-b42b-d42104cc2a36 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:57.898911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bdb5c-33be-47ef-b42b-d42104cc2a36 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:57.899014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bdb5c-33be-47ef-b42b-d42104cc2a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:57.917537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-588bdb5c-33be-47ef-b42b-d42104cc2a36 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:44:57.918714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-588bdb5c-33be-47ef-b42b-d42104cc2a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:44:57.926338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:44:57.962274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8678-d92b-4879-9988-d666128f9615 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:44:57.973189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8678-d92b-4879-9988-d666128f9615 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:44:57.976574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6025c3-743f-4308-acbf-3f960852b018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:44:57.977223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6025c3-743f-4308-acbf-3f960852b018 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:44:57.977498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6025c3-743f-4308-acbf-3f960852b018 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:44:57.977769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6025c3-743f-4308-acbf-3f960852b018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:44:57.986278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6025c3-743f-4308-acbf-3f960852b018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:44:57.987712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6025c3-743f-4308-acbf-3f960852b018 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:44:57.996637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 739/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:00.200119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c7cb9-a6da-4509-9fa9-f7c30dad6d87 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:00.200672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c7cb9-a6da-4509-9fa9-f7c30dad6d87 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:00.200672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c7cb9-a6da-4509-9fa9-f7c30dad6d87 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:00.202023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:00.207220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7786457-03a4-4075-88d4-f12a6befaf17 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:00.207637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7786457-03a4-4075-88d4-f12a6befaf17 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:00.207785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7786457-03a4-4075-88d4-f12a6befaf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:00.224988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b7786457-03a4-4075-88d4-f12a6befaf17 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:00.226098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b7786457-03a4-4075-88d4-f12a6befaf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:00.234087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:45:00.269380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c7cb9-a6da-4509-9fa9-f7c30dad6d87 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:00.279607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c7cb9-a6da-4509-9fa9-f7c30dad6d87 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:00.283316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96604197-88ae-4107-ac10-f82fb72c898d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:00.283932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96604197-88ae-4107-ac10-f82fb72c898d service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:00.284211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96604197-88ae-4107-ac10-f82fb72c898d service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:00.284482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96604197-88ae-4107-ac10-f82fb72c898d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:00.293002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96604197-88ae-4107-ac10-f82fb72c898d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:00.294435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96604197-88ae-4107-ac10-f82fb72c898d service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:00.303888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 730/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:33.485245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d8fc-c246-461a-aa8f-a064ed652136 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:33.485856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d8fc-c246-461a-aa8f-a064ed652136 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:33.485856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d8fc-c246-461a-aa8f-a064ed652136 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:33.487135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 741/1471] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:33.492203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739ebc-77e5-4813-bab8-d31988142868 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:33.492607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739ebc-77e5-4813-bab8-d31988142868 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:33.492796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739ebc-77e5-4813-bab8-d31988142868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:33.510813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d3739ebc-77e5-4813-bab8-d31988142868 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:33.511984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d3739ebc-77e5-4813-bab8-d31988142868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:33.520075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 731/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:45:33.566185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0d4b-8a81-4085-bd08-3419500526c1 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:33.566468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0d4b-8a81-4085-bd08-3419500526c1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:33.566745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c0d4b-8a81-4085-bd08-3419500526c1 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:33.568075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 742/1473] 10.222.0.36 () {54 vars in 962 bytes} [Mon May 16 18:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:33.572832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c84a2-53df-4a94-a22e-69861339735a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:33.573123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c84a2-53df-4a94-a22e-69861339735a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:33.573398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c84a2-53df-4a94-a22e-69861339735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:33.591018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-187c84a2-53df-4a94-a22e-69861339735a None None] MFA Rules not processed for user `fbdba20bd1c243248348d5c5672fefbb`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:33.592176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-187c84a2-53df-4a94-a22e-69861339735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:33.598412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 732/1474] 10.222.0.36 () {58 vars in 1059 bytes} [Mon May 16 18:45:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:45:33.605651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0d4b-8a81-4085-bd08-3419500526c1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:33.615723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c0d4b-8a81-4085-bd08-3419500526c1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:33.619129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad20904-38b4-4b2e-96c8-a5bf05574652 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:33.619737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad20904-38b4-4b2e-96c8-a5bf05574652 service glance] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:33.620017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad20904-38b4-4b2e-96c8-a5bf05574652 service glance] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:33.620291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad20904-38b4-4b2e-96c8-a5bf05574652 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:33.629338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad20904-38b4-4b2e-96c8-a5bf05574652 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:33.630746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad20904-38b4-4b2e-96c8-a5bf05574652 service glance] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:33.639800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 743/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:35.945841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034ed49-9422-440a-8972-1d2df4fb6ac9 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:35.946389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034ed49-9422-440a-8972-1d2df4fb6ac9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:35.946389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034ed49-9422-440a-8972-1d2df4fb6ac9 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:35.947666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 733/1476] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:35.953608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620900d4-d04c-4100-90d8-4913ff853b4b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:35.954155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620900d4-d04c-4100-90d8-4913ff853b4b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:35.954155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620900d4-d04c-4100-90d8-4913ff853b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:35.973119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-620900d4-d04c-4100-90d8-4913ff853b4b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:35.974306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-620900d4-d04c-4100-90d8-4913ff853b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:35.982041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 744/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:45:36.005393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034ed49-9422-440a-8972-1d2df4fb6ac9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:36.018204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034ed49-9422-440a-8972-1d2df4fb6ac9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:36.022636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:36.023273 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:36.023554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:36.023829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:36.024555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:36.026317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:45:36.026317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:45:36.026317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:45:36.027441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93ee3d-cb81-45cb-8399-9bdb848daf37 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:36.047332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 734/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 16 18:45:36 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:45:38.294784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae099e1-775d-4b68-ba3b-34b2e8177510 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:38.295348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae099e1-775d-4b68-ba3b-34b2e8177510 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:38.295348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae099e1-775d-4b68-ba3b-34b2e8177510 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:38.296692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 745/1479] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:38.301908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd4e96-4c6c-493c-8918-c7e870438e8b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:38.302353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd4e96-4c6c-493c-8918-c7e870438e8b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:38.302455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd4e96-4c6c-493c-8918-c7e870438e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:38.319928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-36cd4e96-4c6c-493c-8918-c7e870438e8b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:38.321150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-36cd4e96-4c6c-493c-8918-c7e870438e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:38.329314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 735/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:45:38.371355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae099e1-775d-4b68-ba3b-34b2e8177510 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:38.385191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae099e1-775d-4b68-ba3b-34b2e8177510 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:38.390953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc07494-ff6a-4756-8b01-ca9430329641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:38.391568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc07494-ff6a-4756-8b01-ca9430329641 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:38.391845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc07494-ff6a-4756-8b01-ca9430329641 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:38.392114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc07494-ff6a-4756-8b01-ca9430329641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:38.399849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc07494-ff6a-4756-8b01-ca9430329641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:38.401446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc07494-ff6a-4756-8b01-ca9430329641 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:38.411344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 746/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:40.773691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a28d-af92-45f8-9366-a61f52d131ac None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:40.774281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a28d-af92-45f8-9366-a61f52d131ac None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:40.774281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a28d-af92-45f8-9366-a61f52d131ac None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:40.775562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 736/1482] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:40.780739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274776a-aafc-4c35-8b61-0d99255f17c5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:40.781145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274776a-aafc-4c35-8b61-0d99255f17c5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:40.781324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274776a-aafc-4c35-8b61-0d99255f17c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:40.799708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b274776a-aafc-4c35-8b61-0d99255f17c5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:40.800896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b274776a-aafc-4c35-8b61-0d99255f17c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:40.808391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 747/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:45:40.844920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a28d-af92-45f8-9366-a61f52d131ac None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:40.856246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a28d-af92-45f8-9366-a61f52d131ac None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:40.859812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68031143-c89e-46d3-95e6-c090d6bc8e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:40.860452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68031143-c89e-46d3-95e6-c090d6bc8e53 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:40.860770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68031143-c89e-46d3-95e6-c090d6bc8e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:40.861064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68031143-c89e-46d3-95e6-c090d6bc8e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:40.870102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68031143-c89e-46d3-95e6-c090d6bc8e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:40.871527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68031143-c89e-46d3-95e6-c090d6bc8e53 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:40.880969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 737/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:43.145894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486a8ea-4cc3-4e4e-87e0-813a03a38a7b None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:43.146425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486a8ea-4cc3-4e4e-87e0-813a03a38a7b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:43.146425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486a8ea-4cc3-4e4e-87e0-813a03a38a7b None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:43.147701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 748/1485] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:43.152956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34ba8e-8c63-4992-98ec-3942c3f73fc4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:43.153358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34ba8e-8c63-4992-98ec-3942c3f73fc4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:43.153513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34ba8e-8c63-4992-98ec-3942c3f73fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:43.171384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1d34ba8e-8c63-4992-98ec-3942c3f73fc4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:43.172559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1d34ba8e-8c63-4992-98ec-3942c3f73fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:43.180132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 738/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:45:43.217854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486a8ea-4cc3-4e4e-87e0-813a03a38a7b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:43.228062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486a8ea-4cc3-4e4e-87e0-813a03a38a7b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:43.231473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef44073-2f29-47e6-ace5-0461ff15696b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:43.232085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef44073-2f29-47e6-ace5-0461ff15696b service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:43.232358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef44073-2f29-47e6-ace5-0461ff15696b service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:43.232624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef44073-2f29-47e6-ace5-0461ff15696b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:43.241127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef44073-2f29-47e6-ace5-0461ff15696b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:43.242535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef44073-2f29-47e6-ace5-0461ff15696b service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:43.251923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 749/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:45.669298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df3cd4-d314-4c65-9d8f-9e7a5db469a3 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:45.669874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df3cd4-d314-4c65-9d8f-9e7a5db469a3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:45.669874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df3cd4-d314-4c65-9d8f-9e7a5db469a3 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:45.671175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 739/1488] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:45.676320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f5737-59d7-4181-9aaf-27cc7d362502 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:45.676776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f5737-59d7-4181-9aaf-27cc7d362502 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:45.676912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f5737-59d7-4181-9aaf-27cc7d362502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:45.694459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-580f5737-59d7-4181-9aaf-27cc7d362502 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:45.695631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-580f5737-59d7-4181-9aaf-27cc7d362502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:45.703969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 750/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:45:45.742930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df3cd4-d314-4c65-9d8f-9e7a5db469a3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:45.753646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df3cd4-d314-4c65-9d8f-9e7a5db469a3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:45.757121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0670710-f671-4ed8-9957-542c1ac9ed22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:45.757753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0670710-f671-4ed8-9957-542c1ac9ed22 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:45.758030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0670710-f671-4ed8-9957-542c1ac9ed22 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:45.758298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0670710-f671-4ed8-9957-542c1ac9ed22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:45.767330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0670710-f671-4ed8-9957-542c1ac9ed22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:45.768769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0670710-f671-4ed8-9957-542c1ac9ed22 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:45.778442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 740/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:45:48.146196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b003c-2e41-4c96-99d9-4ed4e4386f10 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:48.146776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b003c-2e41-4c96-99d9-4ed4e4386f10 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:48.146776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b003c-2e41-4c96-99d9-4ed4e4386f10 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:48.148092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 751/1491] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 16 18:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:45:48.153288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983998b-c12c-472a-91a4-9660bcbbe072 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:48.153736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983998b-c12c-472a-91a4-9660bcbbe072 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:48.153840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983998b-c12c-472a-91a4-9660bcbbe072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:48.174371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a983998b-c12c-472a-91a4-9660bcbbe072 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:45:48.175553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a983998b-c12c-472a-91a4-9660bcbbe072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:45:48.183221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 741/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 16 18:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:45:48.221557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b003c-2e41-4c96-99d9-4ed4e4386f10 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:45:48.231794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b003c-2e41-4c96-99d9-4ed4e4386f10 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:45:48.235250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757048f-9de4-4bc8-ae19-70efbc8b0e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:45:48.235865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757048f-9de4-4bc8-ae19-70efbc8b0e79 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:45:48.236145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757048f-9de4-4bc8-ae19-70efbc8b0e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:45:48.236413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757048f-9de4-4bc8-ae19-70efbc8b0e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:45:48.244838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757048f-9de4-4bc8-ae19-70efbc8b0e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:45:48.246239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757048f-9de4-4bc8-ae19-70efbc8b0e79 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:45:48.255175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 752/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:47:15.265472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.266040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.266040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.283276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:47:15.284767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:15.292400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 742/1494] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:47:15.302126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.302616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.302719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.320420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:47:15.321611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:15.329203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 753/1495] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:47:15.333547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.343002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40d7a9-c021-488d-815a-75915f6dd626 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.349423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.350407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.350689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.350963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.351677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:47:15.351982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.353019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.353019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:47:15.353019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:47:15.353891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.365489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 743/1496] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:47:15.374304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.383080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938666fd-1415-48d1-96aa-0acb9f2d0de8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.387527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.388828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.389103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.389373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.390051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.391787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.391787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:47:15.391787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:47:15.392987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.427571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 754/1497] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:47:15 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:47:15.431755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.442061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c7a25-3283-476c-ad5b-1efce4e4ea46 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.494553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.495201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.495479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.495749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.496430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.498301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.498301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:47:15.498301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:47:15.499497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.518758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 744/1498] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:47:15 2022] POST /identity/v3/users => generated 289 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:47:15.522909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.533316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4141b3-766f-49a5-8913-437fab53f4b1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.538044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.538689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.538970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.539241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.539955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.540999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.540999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:47:15.540999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:47:15.541877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.554830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 755/1499] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:47:15.558688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.568141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ce86-ab5b-473c-b88b-d695d368a309 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.572527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.573199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.573482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.573756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.574466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.575454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.575454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:47:15.575454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:47:15.576331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f658e-cf41-456c-93d1-8423d69056f9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.588981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 745/1500] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:47:15.593326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.602935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beeb72-1c38-452e-8800-819294ebccd7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.608137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.609053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.609337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.609614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] PATH_INFO: `/v3/projects/9d231aede578446e8e194f46285434a3/users/0b6b43d68e4546c684e6d4305cf207a8/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.623513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d231aede578446e8e194f46285434a3, user_id=0b6b43d68e4546c684e6d4305cf207a8, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.624543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.624543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:47:15.624543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:47:15.626484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177d576-b62a-4ff7-9798-64f089b8b484 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.639250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 756/1501] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:47:15 2022] PUT /identity/v3/projects/9d231aede578446e8e194f46285434a3/users/0b6b43d68e4546c684e6d4305cf207a8/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:47:15.650461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d0dad-ef04-472d-8913-b05f3979a92e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.650751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d0dad-ef04-472d-8913-b05f3979a92e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.651027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d0dad-ef04-472d-8913-b05f3979a92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.677257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6c3d0dad-ef04-472d-8913-b05f3979a92e None None] MFA Rules not processed for user `0b6b43d68e4546c684e6d4305cf207a8`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:47:15.678452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6c3d0dad-ef04-472d-8913-b05f3979a92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:15.737343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 746/1502] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:47:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:47:15.750568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f810e-d926-4171-bf37-489cdc0088dc None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.750861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f810e-d926-4171-bf37-489cdc0088dc None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.751141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f810e-d926-4171-bf37-489cdc0088dc None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.752517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 757/1503] 10.222.0.36 () {54 vars in 963 bytes} [Mon May 16 18:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:47:15.757480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681640e-6f3b-4a5d-bbf4-519ace5d4f63 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.757769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681640e-6f3b-4a5d-bbf4-519ace5d4f63 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.758040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681640e-6f3b-4a5d-bbf4-519ace5d4f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.775556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1681640e-6f3b-4a5d-bbf4-519ace5d4f63 None None] MFA Rules not processed for user `dfc10bc677cc4cf4b673fdcbd937d352`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:47:15.776758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1681640e-6f3b-4a5d-bbf4-519ace5d4f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:15.824816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 747/1504] 10.222.0.36 () {58 vars in 1060 bytes} [Mon May 16 18:47:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:47:15.832781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f810e-d926-4171-bf37-489cdc0088dc None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.843676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f810e-d926-4171-bf37-489cdc0088dc None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.847140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.847763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.848046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.848320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.856366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.858322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:15.866859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 758/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:47:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:47:15.907859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756776db-c403-4238-b2e7-c3fe0da0aeac None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.908144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756776db-c403-4238-b2e7-c3fe0da0aeac None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.908437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756776db-c403-4238-b2e7-c3fe0da0aeac None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.910011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 748/1506] 10.222.0.36 () {52 vars in 829 bytes} [Mon May 16 18:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:47:15.913667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:15.972558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f96c3-4cfb-4f76-bf1c-972233920d26 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:15.976998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:15.977861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:15.978141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:15.978411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] PATH_INFO: `/v3/users/0b6b43d68e4546c684e6d4305cf207a8` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:15.979599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6b43d68e4546c684e6d4305cf207a8)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:15.980607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:15.980607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:47:15.980607 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:47:15.982306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:16.051552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6b43d68e4546c684e6d4305cf207a8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.064142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6b43d68e4546c684e6d4305cf207a8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.065045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6b43d68e4546c684e6d4305cf207a8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.073706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6b43d68e4546c684e6d4305cf207a8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.074395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6b43d68e4546c684e6d4305cf207a8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.096886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 759/1507] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:47:15 2022] DELETE /identity/v3/users/0b6b43d68e4546c684e6d4305cf207a8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:47:16.101570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:16.101885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:16.102233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:16.121214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:47:16.123144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:16.179345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 749/1508] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:47:16.188878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:16.190269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:16.244193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edd3c8-ce59-401e-ad53-baf83d767801 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:16.247812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05136007-b06c-40db-88e0-28428810f981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:16.248447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05136007-b06c-40db-88e0-28428810f981 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:16.248759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05136007-b06c-40db-88e0-28428810f981 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:16.249040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05136007-b06c-40db-88e0-28428810f981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:16.257609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05136007-b06c-40db-88e0-28428810f981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:16.259036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05136007-b06c-40db-88e0-28428810f981 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:16.268077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 760/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:47:16.642284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:47:16.644169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:47:16.656615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32281ace-f366-430e-bc28-21b4bba8b9c9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:47:16.661514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:47:16.662382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:47:16.662687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:47:16.662928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] PATH_INFO: `/v3/projects/9d231aede578446e8e194f46285434a3` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:47:16.664144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d231aede578446e8e194f46285434a3)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:47:16.666606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:47:16.666606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:47:16.666606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:47:16.666606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:47:16.725296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d231aede578446e8e194f46285434a3', 'request_id': 'req-4cb2c99c-0f00-4633-be63-328eab26f154'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.731499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d231aede578446e8e194f46285434a3', 'request_id': 'req-4cb2c99c-0f00-4633-be63-328eab26f154'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.741050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d231aede578446e8e194f46285434a3', 'request_id': 'req-4cb2c99c-0f00-4633-be63-328eab26f154'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.758608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] The token cache is being invalidate because project 9d231aede578446e8e194f46285434a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:47:16.758888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.759605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4cb2c99c-0f00-4633-be63-328eab26f154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:47:16.761287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 750/1510] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:47:16 2022] DELETE /identity/v3/projects/9d231aede578446e8e194f46285434a3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:48:56.678096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be1106-a9a0-4878-a43e-a50a941aba2d None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:56.678693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be1106-a9a0-4878-a43e-a50a941aba2d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:56.678693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be1106-a9a0-4878-a43e-a50a941aba2d None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:56.680006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:48:56.685042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ab8a3-3de5-45b4-ad8d-70f2808285d2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:56.685444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ab8a3-3de5-45b4-ad8d-70f2808285d2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:56.685592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ab8a3-3de5-45b4-ad8d-70f2808285d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:56.703092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-ed0ab8a3-3de5-45b4-ad8d-70f2808285d2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:48:56.704282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ed0ab8a3-3de5-45b4-ad8d-70f2808285d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:48:56.759083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:48:56.965408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be1106-a9a0-4878-a43e-a50a941aba2d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:48:56.967631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e7be1106-a9a0-4878-a43e-a50a941aba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:48:57.044550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be1106-a9a0-4878-a43e-a50a941aba2d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:48:57.050655 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054b6a7-6389-4787-bf6a-2a43521834cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:48:57.051295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054b6a7-6389-4787-bf6a-2a43521834cc service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:57.051572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054b6a7-6389-4787-bf6a-2a43521834cc service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:57.051844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054b6a7-6389-4787-bf6a-2a43521834cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:57.059731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b054b6a7-6389-4787-bf6a-2a43521834cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:48:57.061371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b054b6a7-6389-4787-bf6a-2a43521834cc service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:48:57.070982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 762/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:48:57.975968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa5c4a-721c-421a-a5d9-5452c787c273 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:57.976427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa5c4a-721c-421a-a5d9-5452c787c273 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:57.976539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa5c4a-721c-421a-a5d9-5452c787c273 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:57.977917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:48:57.982721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b18ec-c98c-4a01-b2c3-1382e381807b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:57.983153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b18ec-c98c-4a01-b2c3-1382e381807b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:57.983262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b18ec-c98c-4a01-b2c3-1382e381807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:58.001386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-952b18ec-c98c-4a01-b2c3-1382e381807b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:48:58.002577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-952b18ec-c98c-4a01-b2c3-1382e381807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:48:58.010289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:48:58.162627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa5c4a-721c-421a-a5d9-5452c787c273 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:48:58.177054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa5c4a-721c-421a-a5d9-5452c787c273 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:48:58.182433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10456246-935a-444b-ba3f-df69dfefaadd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:48:58.183097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10456246-935a-444b-ba3f-df69dfefaadd service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:58.183377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10456246-935a-444b-ba3f-df69dfefaadd service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:58.183645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10456246-935a-444b-ba3f-df69dfefaadd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:58.191626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10456246-935a-444b-ba3f-df69dfefaadd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:48:58.193229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10456246-935a-444b-ba3f-df69dfefaadd service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:48:58.202243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 753/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:48:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:48:58.339409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acb9d0-f598-419c-96f0-a9adab34e5eb None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:58.339699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acb9d0-f598-419c-96f0-a9adab34e5eb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:58.340034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acb9d0-f598-419c-96f0-a9adab34e5eb None None] PATH_INFO: `/v3` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:58.341372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 764/1517] 10.222.0.59 () {54 vars in 925 bytes} [Mon May 16 18:48:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:48:58.348412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d34fef-e643-4cd9-999d-b80bff925971 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:48:58.348724 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d34fef-e643-4cd9-999d-b80bff925971 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:48:58.349008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d34fef-e643-4cd9-999d-b80bff925971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:48:58.366716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-50d34fef-e643-4cd9-999d-b80bff925971 None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:48:58.367898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-50d34fef-e643-4cd9-999d-b80bff925971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:48:58.427574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 754/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Mon May 16 18:48:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:07.746723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2a849-379e-46b9-84f0-8184fc4f5e7c None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:07.747205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2a849-379e-46b9-84f0-8184fc4f5e7c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:07.747299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2a849-379e-46b9-84f0-8184fc4f5e7c None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:07.748617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 765/1519] 10.222.0.36 () {54 vars in 960 bytes} [Mon May 16 18:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:49:07.753661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04567-610c-4396-a541-5f6521d772fb None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:07.754603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04567-610c-4396-a541-5f6521d772fb None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:07.754712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04567-610c-4396-a541-5f6521d772fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:07.772581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-28c04567-610c-4396-a541-5f6521d772fb None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:07.773780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-28c04567-610c-4396-a541-5f6521d772fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:07.782317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 755/1520] 10.222.0.36 () {58 vars in 1057 bytes} [Mon May 16 18:49:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:07.791188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2a849-379e-46b9-84f0-8184fc4f5e7c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:07.804154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2a849-379e-46b9-84f0-8184fc4f5e7c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:07.808985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541165e-4712-468d-bb11-62b8a75b72c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:07.809602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541165e-4712-468d-bb11-62b8a75b72c4 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:07.809886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541165e-4712-468d-bb11-62b8a75b72c4 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:07.810156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541165e-4712-468d-bb11-62b8a75b72c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:07.818623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541165e-4712-468d-bb11-62b8a75b72c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:07.820183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541165e-4712-468d-bb11-62b8a75b72c4 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:07.829483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 766/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:49:09.510377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296812-fc2c-4e0a-a2e7-d4c53ec4d0a4 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:09.510971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296812-fc2c-4e0a-a2e7-d4c53ec4d0a4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:09.510971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f296812-fc2c-4e0a-a2e7-d4c53ec4d0a4 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:09.512261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:49:09.517386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3def7-2b53-4954-8a70-c6da9ed9dd06 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:09.517828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3def7-2b53-4954-8a70-c6da9ed9dd06 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:09.517947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3def7-2b53-4954-8a70-c6da9ed9dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:09.539306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0df3def7-2b53-4954-8a70-c6da9ed9dd06 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:09.540510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0df3def7-2b53-4954-8a70-c6da9ed9dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:09.549748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:49:09.700504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f296812-fc2c-4e0a-a2e7-d4c53ec4d0a4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:09.714530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f296812-fc2c-4e0a-a2e7-d4c53ec4d0a4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:09.719798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04335da-32a4-4567-851d-db56fda84c11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:09.720422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04335da-32a4-4567-851d-db56fda84c11 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:09.720735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04335da-32a4-4567-851d-db56fda84c11 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:09.721012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04335da-32a4-4567-851d-db56fda84c11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:09.728862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04335da-32a4-4567-851d-db56fda84c11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:09.730448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04335da-32a4-4567-851d-db56fda84c11 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:09.740221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 757/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:49:10.623130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862200d4-a777-4177-9a15-1518baa8ecea None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:10.623729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862200d4-a777-4177-9a15-1518baa8ecea None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:10.623729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862200d4-a777-4177-9a15-1518baa8ecea None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:10.625080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 16 18:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:49:10.629805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735c2ff-0149-41fc-86fa-cb021e9af081 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:10.630233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735c2ff-0149-41fc-86fa-cb021e9af081 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:10.630359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735c2ff-0149-41fc-86fa-cb021e9af081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:10.648173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4735c2ff-0149-41fc-86fa-cb021e9af081 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:10.649411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4735c2ff-0149-41fc-86fa-cb021e9af081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:10.657019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 16 18:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:10.805966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862200d4-a777-4177-9a15-1518baa8ecea None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:10.820165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862200d4-a777-4177-9a15-1518baa8ecea None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:10.825608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e5280-e392-4585-ac0f-da011c9f852b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:10.826237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e5280-e392-4585-ac0f-da011c9f852b service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:10.826519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e5280-e392-4585-ac0f-da011c9f852b service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:10.826796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e5280-e392-4585-ac0f-da011c9f852b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:10.834663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e5280-e392-4585-ac0f-da011c9f852b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:10.836244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e5280-e392-4585-ac0f-da011c9f852b service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:10.845416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 769/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:49:14.971352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1cb0e-4569-474a-bf85-b2a2fbde1a9e None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:14.971773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1cb0e-4569-474a-bf85-b2a2fbde1a9e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:14.971896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1cb0e-4569-474a-bf85-b2a2fbde1a9e None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:14.973896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:49:14.978879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e4135-ec8f-4cdb-8b5d-4a9eac828b46 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:14.979328 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e4135-ec8f-4cdb-8b5d-4a9eac828b46 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:14.979423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e4135-ec8f-4cdb-8b5d-4a9eac828b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:14.996728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ac0e4135-ec8f-4cdb-8b5d-4a9eac828b46 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:14.997900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ac0e4135-ec8f-4cdb-8b5d-4a9eac828b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:15.005568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:15.042182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1cb0e-4569-474a-bf85-b2a2fbde1a9e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:15.043860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-55a1cb0e-4569-474a-bf85-b2a2fbde1a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:15.099927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1cb0e-4569-474a-bf85-b2a2fbde1a9e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:15.103404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e3c7a-f9c9-417a-b3c6-9f1d45da24ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:15.104029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3c7a-f9c9-417a-b3c6-9f1d45da24ce service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:15.104309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3c7a-f9c9-417a-b3c6-9f1d45da24ce service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:15.104583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3c7a-f9c9-417a-b3c6-9f1d45da24ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:15.114634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e3c7a-f9c9-417a-b3c6-9f1d45da24ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:15.116072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e3c7a-f9c9-417a-b3c6-9f1d45da24ce service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:15.127000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 760/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:49:17.337197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d897b-65fb-4e3e-9f0d-702c89379cf6 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:17.337730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d897b-65fb-4e3e-9f0d-702c89379cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:17.337730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d897b-65fb-4e3e-9f0d-702c89379cf6 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:17.339128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:49:17.344368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900103ae-4613-4838-84b3-c42b5c12c40c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:17.344828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900103ae-4613-4838-84b3-c42b5c12c40c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:17.344965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900103ae-4613-4838-84b3-c42b5c12c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:17.365022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-900103ae-4613-4838-84b3-c42b5c12c40c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:17.366219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-900103ae-4613-4838-84b3-c42b5c12c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:17.373797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:17.408319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d897b-65fb-4e3e-9f0d-702c89379cf6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:17.419531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d897b-65fb-4e3e-9f0d-702c89379cf6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:17.423012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c39b-bbe4-400f-8700-a066e20e22a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:17.423635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c39b-bbe4-400f-8700-a066e20e22a4 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:17.423911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c39b-bbe4-400f-8700-a066e20e22a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:17.424182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c39b-bbe4-400f-8700-a066e20e22a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:17.432678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665c39b-bbe4-400f-8700-a066e20e22a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:17.434107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665c39b-bbe4-400f-8700-a066e20e22a4 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:17.443600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 772/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:49:21.124846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb711-cc13-471b-a7fd-7613eebe9f18 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:21.125458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb711-cc13-471b-a7fd-7613eebe9f18 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:21.125458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb711-cc13-471b-a7fd-7613eebe9f18 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:21.126740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:49:21.132662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e578197-0f62-4ebe-bd89-8fbe2bc86f31 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:21.133187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e578197-0f62-4ebe-bd89-8fbe2bc86f31 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:21.133300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e578197-0f62-4ebe-bd89-8fbe2bc86f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:21.150646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0e578197-0f62-4ebe-bd89-8fbe2bc86f31 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:21.151866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0e578197-0f62-4ebe-bd89-8fbe2bc86f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:21.160491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:21.437948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb711-cc13-471b-a7fd-7613eebe9f18 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:21.439634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-859fb711-cc13-471b-a7fd-7613eebe9f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:21.494797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb711-cc13-471b-a7fd-7613eebe9f18 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:21.498264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b4c16-d855-4176-ac21-60ac493cbe50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:21.498883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b4c16-d855-4176-ac21-60ac493cbe50 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:21.499161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b4c16-d855-4176-ac21-60ac493cbe50 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:21.499430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b4c16-d855-4176-ac21-60ac493cbe50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:21.508423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b4c16-d855-4176-ac21-60ac493cbe50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:21.509852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b4c16-d855-4176-ac21-60ac493cbe50 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:21.518791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 763/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:49:23.780597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e31e5-efab-4927-823e-1c84e0b01638 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:23.781219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e31e5-efab-4927-823e-1c84e0b01638 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:23.781219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e31e5-efab-4927-823e-1c84e0b01638 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:23.782552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 16 18:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:49:23.788074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd646-fe6f-4d44-94e4-608d14649de3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:23.788509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd646-fe6f-4d44-94e4-608d14649de3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:23.788652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd646-fe6f-4d44-94e4-608d14649de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:23.806673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-939fd646-fe6f-4d44-94e4-608d14649de3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:49:23.807839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-939fd646-fe6f-4d44-94e4-608d14649de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:23.815471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 16 18:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:49:24.083943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e31e5-efab-4927-823e-1c84e0b01638 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:49:24.085658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bd4e31e5-efab-4927-823e-1c84e0b01638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:49:24.098845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e31e5-efab-4927-823e-1c84e0b01638 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:49:24.102279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4d151-c22b-4f48-8860-9615dcd6faa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:49:24.102900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4d151-c22b-4f48-8860-9615dcd6faa3 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:49:24.103178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4d151-c22b-4f48-8860-9615dcd6faa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:49:24.103452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4d151-c22b-4f48-8860-9615dcd6faa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:49:24.111956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4d151-c22b-4f48-8860-9615dcd6faa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:49:24.113430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4d151-c22b-4f48-8860-9615dcd6faa3 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:49:24.122466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 775/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:53.969819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:53.970896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:53.971392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.002848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.004029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.015149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 765/1540] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.025227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.025839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.025839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.061054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.062267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.086873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 776/1541] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.091637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.105862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4193e31-b921-4ff9-8f82-d05df0ea79cc None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.116434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.117184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.118022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.118300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.119022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:54.119376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.121186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.137597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 766/1542] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:54.145439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.155152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd347-f3c0-4649-8ada-52e9f2b3e88a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.165191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.165852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.166133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.166410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.167101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.169047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.212797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 777/1543] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:54.216992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.233827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e240a2c-756c-42d0-87d2-2087ee8bd437 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.332016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c1cf-9739-4c37-a57e-2f5c761befcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.332736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c1cf-9739-4c37-a57e-2f5c761befcc admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.333131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c1cf-9739-4c37-a57e-2f5c761befcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.333410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c1cf-9739-4c37-a57e-2f5c761befcc admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.334116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c1cf-9739-4c37-a57e-2f5c761befcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.336075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c1cf-9739-4c37-a57e-2f5c761befcc admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.370676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 767/1544] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:54.375547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.395016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1145c-5143-4395-b857-d37a2abffd4d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.395627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.395915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.396328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.410649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f88b7-e6bd-4c48-a831-00b231415142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.411348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f88b7-e6bd-4c48-a831-00b231415142 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.411629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f88b7-e6bd-4c48-a831-00b231415142 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.411905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f88b7-e6bd-4c48-a831-00b231415142 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.412651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f88b7-e6bd-4c48-a831-00b231415142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.414799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f88b7-e6bd-4c48-a831-00b231415142 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.422648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.423853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.435588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 778/1545] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:54.437282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.437574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.437855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.438199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 768/1546] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:50:54.439033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.453835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b48c2-2c90-458b-a66e-b63382bfbfc9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.464432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a209ff4-09c1-41af-8408-171d44b1665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.465135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a209ff4-09c1-41af-8408-171d44b1665d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.465412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a209ff4-09c1-41af-8408-171d44b1665d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.465684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a209ff4-09c1-41af-8408-171d44b1665d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.466082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.466406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a209ff4-09c1-41af-8408-171d44b1665d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.467729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.468013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a209ff4-09c1-41af-8408-171d44b1665d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.483810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 769/1547] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:54.484494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 779/1548] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.485133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549afe-90a5-4fc8-9425-27ead8d41500 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.485417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549afe-90a5-4fc8-9425-27ead8d41500 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.485751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549afe-90a5-4fc8-9425-27ead8d41500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.489158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.502732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310edc6a-0ff7-43e9-a527-1f7d2d22e416 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.513415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cf196-43e6-45ea-b756-62a33978d0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.514322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cf196-43e6-45ea-b756-62a33978d0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.514601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cf196-43e6-45ea-b756-62a33978d0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.514874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cf196-43e6-45ea-b756-62a33978d0d2 admin admin] PATH_INFO: `/v3/projects/8ebb54e66dec4709ad23a05f4d793801/users/8934281a28e24f93a058b4e99738014e/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.522347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-ff549afe-90a5-4fc8-9425-27ead8d41500 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.523592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ff549afe-90a5-4fc8-9425-27ead8d41500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.530806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cf196-43e6-45ea-b756-62a33978d0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebb54e66dec4709ad23a05f4d793801, user_id=8934281a28e24f93a058b4e99738014e, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.533494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cf196-43e6-45ea-b756-62a33978d0d2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.542160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 770/1549] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.543808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.544095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.544370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.580773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 780/1550] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:54 2022] PUT /identity/v3/projects/8ebb54e66dec4709ad23a05f4d793801/users/8934281a28e24f93a058b4e99738014e/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:50:54.582211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4af451-87e5-4b0a-96fb-38b372396be1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.582507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4af451-87e5-4b0a-96fb-38b372396be1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.582794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4af451-87e5-4b0a-96fb-38b372396be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.585424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.586699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.615759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6a4af451-87e5-4b0a-96fb-38b372396be1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.617018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6a4af451-87e5-4b0a-96fb-38b372396be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.676275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 771/1551] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.677481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.687196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ba7d3-aaad-477a-825b-f6e762ee7ea8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.691148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 781/1552] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.695245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.695245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.695245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.695245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.695744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335be2-145f-47fc-9f74-39d006bfc33e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.695838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:54.695934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335be2-145f-47fc-9f74-39d006bfc33e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.695934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335be2-145f-47fc-9f74-39d006bfc33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.696100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.696100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558042f-c8fb-42f9-a27d-2f9b36762973 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.710967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 772/1553] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:54.712247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d13ed-a55a-4255-85e1-ab1379edea2f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.712663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d13ed-a55a-4255-85e1-ab1379edea2f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.714244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d13ed-a55a-4255-85e1-ab1379edea2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.716690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-72335be2-145f-47fc-9f74-39d006bfc33e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.717939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-72335be2-145f-47fc-9f74-39d006bfc33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.726255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 782/1554] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.729994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f45d9-1fa3-48b3-b930-ef303076c21a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.730289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f45d9-1fa3-48b3-b930-ef303076c21a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.730570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f45d9-1fa3-48b3-b930-ef303076c21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.740914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-719d13ed-a55a-4255-85e1-ab1379edea2f None None] MFA Rules not processed for user `8934281a28e24f93a058b4e99738014e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.742033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-719d13ed-a55a-4255-85e1-ab1379edea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.754350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-447f45d9-1fa3-48b3-b930-ef303076c21a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.755550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-447f45d9-1fa3-48b3-b930-ef303076c21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.764282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 783/1555] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.765557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.765854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.766128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.783979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.785453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.793661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 784/1556] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.794440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.803301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95dcd-833a-4d70-a627-9d6a0040f835 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.805656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 773/1557] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.806917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0263ec-a96f-4274-ad3e-5aa3ba05b039 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.807200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0263ec-a96f-4274-ad3e-5aa3ba05b039 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.807599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0263ec-a96f-4274-ad3e-5aa3ba05b039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.807944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.808581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.809071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.809348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.810056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:54.810359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.811362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 16 18:50:54.811362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:50:54.811362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:50:54.812224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.830386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 785/1558] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:54.830386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.830697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3f0263ec-a96f-4274-ad3e-5aa3ba05b039 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.830697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3f0263ec-a96f-4274-ad3e-5aa3ba05b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.833889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06eb78-0148-4cf1-bd0c-e531115537a3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.835877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 774/1559] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.837622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaedbe-84bf-43de-b622-59fb182f4ae6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.837915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaedbe-84bf-43de-b622-59fb182f4ae6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.838172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaedbe-84bf-43de-b622-59fb182f4ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.842533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d92076-3117-489f-a725-c5f327bf138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.843201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d92076-3117-489f-a725-c5f327bf138a admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.843482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d92076-3117-489f-a725-c5f327bf138a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.843755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d92076-3117-489f-a725-c5f327bf138a admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.844426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d92076-3117-489f-a725-c5f327bf138a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.846409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d92076-3117-489f-a725-c5f327bf138a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.856402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-93eaedbe-84bf-43de-b622-59fb182f4ae6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.857623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-93eaedbe-84bf-43de-b622-59fb182f4ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.865830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 775/1560] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.867208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b6012-6c36-41a2-8dbb-d9bd5dfd7eb0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.867333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b6012-6c36-41a2-8dbb-d9bd5dfd7eb0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.867622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b6012-6c36-41a2-8dbb-d9bd5dfd7eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.875879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 786/1561] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:54.875879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.875879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.875879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.884549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-435b6012-6c36-41a2-8dbb-d9bd5dfd7eb0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.886445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-435b6012-6c36-41a2-8dbb-d9bd5dfd7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.892028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.893208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.945013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 776/1562] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.946409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.946694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.946965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.949909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 787/1563] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.950724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.960036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09eb87-f195-4368-a7f3-1868783b9241 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.965428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.966083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.966364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.966635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.967314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.969274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:54.969680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:54.970872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:54.979508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 777/1564] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:54.980104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.989653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1954a-2861-4221-9f7b-abf4a2d32024 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:54.994720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:54.995340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:54.995578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:54.995849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:54.996526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:54.997213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 788/1565] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:54.997569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:50:54.997569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:50:54.997569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:50:54.998034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:54.998834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.023527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 778/1566] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:54 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.024861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.056771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b51c4-07e1-41d0-8e27-7faab07560c9 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.083769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e6336-5973-42dc-b916-ab701d252527 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.088868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.089520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.089851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.090129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.090850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:55.091154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.092756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.104160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 779/1567] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:55.105026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.110189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.110917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.111231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.111515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.112225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:55.112528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.114307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.116387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeaf00-4b3b-482f-9588-813e17ca6cad admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.123047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431137ea-879a-4ceb-b509-8b3752dc3090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.123457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431137ea-879a-4ceb-b509-8b3752dc3090 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.123833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431137ea-879a-4ceb-b509-8b3752dc3090 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.124025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431137ea-879a-4ceb-b509-8b3752dc3090 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.124744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431137ea-879a-4ceb-b509-8b3752dc3090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.126870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431137ea-879a-4ceb-b509-8b3752dc3090 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.129057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 789/1568] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:55.129800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.140728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9825c6-9985-4946-9679-49bc525f291a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.145835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.146791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.146791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.147002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.147793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:55.148090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.150308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.150984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 780/1569] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.152337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.153042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.153042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.169641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 790/1570] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:55.169641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.172844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df33112-1297-4197-8631-2c72c24df5d2 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.175466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:55.176662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:55.177644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.178280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.178561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.178833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.179537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:55.179842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.181537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.184453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 781/1571] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:55.185231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.192926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 791/1572] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:55.193689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.193988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3644-8547-40b6-a78f-ae4b5188e2e7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.201617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.202255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.202532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.202801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.203474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.205434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.207295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534b53-8207-4529-aa1a-ee7644d1304b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.212517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.213371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.214100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.214383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.215446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.216450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:50:55.216450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:50:55.216450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:50:55.217772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.225952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 782/1573] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.226808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.238744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45440da9-fad8-4537-90a1-7d402862f4db admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.240265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 792/1574] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.241140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.244528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.245301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.245535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.245926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.246594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.248502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.250336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c325e-a3c3-4798-9788-62ff7c4bdfe3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.254841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.255462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.255742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.256017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.256710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.259399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.275025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 783/1575] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.276125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.288074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 793/1576] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.288074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.335141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1952c7-51f3-4cf9-9949-5b5eb1252769 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.340447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.341142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.341423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.341695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.342412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.343992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.344282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f171f8-7cbe-46c4-aeb9-56557184b8a8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.348846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.349500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.349781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.350059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.350743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.352674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.353476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 784/1577] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.354294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.363224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdee98a-cd8f-4e00-a3f7-22c240f7e9bb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.367849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.368528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.368864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.369205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.369919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:50:55.370225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.371899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.380289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 794/1578] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.381113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.391386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 785/1579] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:50:55.391386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.396837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16914e03-546e-4fcd-a574-b0e7b98bca0f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.447736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97182be6-9761-4c0c-9e9b-cc876816d7bd admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.452648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.453306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.453590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.453868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.454590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.456506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.470164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.470941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.471218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.471492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.472208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.474304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.486242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 786/1580] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.491391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 795/1581] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.491560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.491659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.500157 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e48fc1-b5bb-4a27-a2f3-0dffbd6f7d2d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.563718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f055abc-694f-4b54-bf2d-12d5fc51a60d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.564797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.565465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.565878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.566151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.566865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.568450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.575992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.576837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.577120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.577395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.578084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.579237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 787/1582] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.580020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.580150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.589810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d14562-5d80-4f83-b07a-9301d1cbb197 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.595086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.595761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.596041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.596310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.597025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.598920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.600214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 796/1583] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.601077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.611778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287087dc-865b-4c4d-b46c-2dc7626e2987 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.617327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.618499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.618805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.619076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.619797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.620017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 788/1584] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.621047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.621787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.630854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d02a29-fde2-478b-b9ba-7a56153af6df admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.636814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 797/1585] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.636814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.639102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.639766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.640048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.640322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.641041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.642950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.647199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc25021-eebd-4a22-b326-969342b02472 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.652251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.652965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.653558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.653872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.654732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.656820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.667244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 789/1586] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.668146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.678131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b8f0-4549-43f0-8ff0-9f8fc5e6798d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.684338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.685452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.685733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.686007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] PATH_INFO: `/v3/projects/429270ef798f48859eb7acbd41ce89f0/users/8327063b27d34d7f9da9aeae6c4bd198/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.690395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 798/1587] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.691337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.703976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=429270ef798f48859eb7acbd41ce89f0, user_id=8327063b27d34d7f9da9aeae6c4bd198, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.705087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:50:55.705087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:50:55.705087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:50:55.707045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.721292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 790/1588] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:55 2022] PUT /identity/v3/projects/429270ef798f48859eb7acbd41ce89f0/users/8327063b27d34d7f9da9aeae6c4bd198/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:55.721952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.768768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda1824-08e9-4fd2-a373-bf15f6d7d9b5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.785312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af0c7f-e5d9-4778-88c4-0390c0098537 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.790016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.790669 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.790948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.791221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.791934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.793531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.803025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 791/1589] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.803025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.813846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ee600-e4bd-4799-b6c0-f1383b6a265a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.818962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.819592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.819898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.820178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.821571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.822642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.823111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.823303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.823586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.823854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.824530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.826469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.832673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 792/1590] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.832673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.842853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330f4ce-4850-43ee-bc97-133e917bdef3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.846697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 799/1591] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.847342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.850022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.850703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.850982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.851256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.852001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.853592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.858488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3ce5c-6934-4aaf-9c46-cfc9aa880f14 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.863057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.863767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 793/1592] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.864701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.867122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.867930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.868219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] PATH_INFO: `/v3/projects/96b041b622fb4e0fa52ae670ba7b9505/users/167f6e68bc0e4cc9bf551c9ad1430bb7/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.874400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76864599-e548-4868-a6aa-b2766c6c20b3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.879129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.879766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.880043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.880448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.881192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.882746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.883742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b041b622fb4e0fa52ae670ba7b9505, user_id=167f6e68bc0e4cc9bf551c9ad1430bb7, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.887993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.893059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 794/1593] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.894118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.903276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 800/1594] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:55 2022] PUT /identity/v3/projects/96b041b622fb4e0fa52ae670ba7b9505/users/167f6e68bc0e4cc9bf551c9ad1430bb7/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:55.904135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aa52d-e793-4aee-ad5e-20b3848aae06 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.904315 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.960316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.960930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.961233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.961505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.962184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.964194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.970316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394f20f-ec8e-4196-919e-181cb00855dc admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:55.975012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:55.975713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:55.976027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:55.976331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:55.977697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:55.979339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:55.984404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 795/1595] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:55 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:55.985181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.995585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 801/1596] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:55.995585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:55.998005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5292cc-fbad-42d1-a645-876ee2c270a0 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.002712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.002848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76562-fa1e-4829-ae73-4a92d828ad2f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.003572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.003855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.004127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] PATH_INFO: `/v3/projects/c8c41c631a51459199afd424bd8de81c/users/3b39f00235c34612b591cd2d974d9969/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.007317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.007941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.008253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.008529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.009275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.010856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.018582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c41c631a51459199afd424bd8de81c, user_id=3b39f00235c34612b591cd2d974d9969, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.020390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 802/1597] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.021147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.022018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.032015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb483a70-1459-46b0-a683-d9c1005f071f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.035611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 796/1598] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:55 2022] PUT /identity/v3/projects/c8c41c631a51459199afd424bd8de81c/users/3b39f00235c34612b591cd2d974d9969/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.039807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.087297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.087958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.088235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.088506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.089255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.090837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.094146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f40df-919e-4e07-aa58-633eb492832c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.098634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22c59b-917e-4d01-8c88-87ef3637f077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.099285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22c59b-917e-4d01-8c88-87ef3637f077 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.099561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22c59b-917e-4d01-8c88-87ef3637f077 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.099829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22c59b-917e-4d01-8c88-87ef3637f077 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.100313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 803/1599] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.100540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22c59b-917e-4d01-8c88-87ef3637f077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.101159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.102666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22c59b-917e-4d01-8c88-87ef3637f077 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.112180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e570a21-ef3e-4c0e-86e7-c14bfd30add5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.112368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 797/1600] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.113631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.113912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.114240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.120895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.121423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.121714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.121988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.122709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.124304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.133353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.137340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 804/1601] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.137494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.137613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.143510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 798/1602] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.144238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.146511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5e57-8794-4fa0-a7d3-761f8b40c746 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.151662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.152341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.152623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.152932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.153167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbea7dd-1149-40a6-8360-69281f1e4833 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.153650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.155237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.158091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69a117-43bd-44a4-b0e6-7017088dcead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.158973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69a117-43bd-44a4-b0e6-7017088dcead admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.159246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69a117-43bd-44a4-b0e6-7017088dcead admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.159762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69a117-43bd-44a4-b0e6-7017088dcead admin admin] PATH_INFO: `/v3/projects/429270ef798f48859eb7acbd41ce89f0/users/8327063b27d34d7f9da9aeae6c4bd198/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.161710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69a117-43bd-44a4-b0e6-7017088dcead admin admin] RBAC: Authorizing `identity:create_grant(project_id=429270ef798f48859eb7acbd41ce89f0, user_id=8327063b27d34d7f9da9aeae6c4bd198, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.164943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69a117-43bd-44a4-b0e6-7017088dcead admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.166440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 805/1603] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.167199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.175345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 799/1604] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:56 2022] PUT /identity/v3/projects/429270ef798f48859eb7acbd41ce89f0/users/8327063b27d34d7f9da9aeae6c4bd198/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.176330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82947e-63b5-4dcd-a700-2bd9072800f0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.177136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.177429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.177707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.195190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.196386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.230807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.231713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.232319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.232319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] PATH_INFO: `/v3/projects/69a6740a82ca4787a113f960eb794573/users/cbef2598731b4cd9abfd0c8a3bc3cdbe/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.248137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a6740a82ca4787a113f960eb794573, user_id=cbef2598731b4cd9abfd0c8a3bc3cdbe, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.251844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.252421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 800/1605] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.255674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.262836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8775329-cf9b-4897-a8a2-696cc939efe6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.266787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 806/1606] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:56 2022] PUT /identity/v3/projects/69a6740a82ca4787a113f960eb794573/users/cbef2598731b4cd9abfd0c8a3bc3cdbe/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.267590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.316476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.317175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.317456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.317737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.318453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.320051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.329489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 801/1607] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.330311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.334543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1314731-c9de-4201-9fd3-c8abfcb6bf1d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.339694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.340093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7ad43-0466-41ac-b58c-2f0e0f52a750 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.340571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.340883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.341158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] PATH_INFO: `/v3/projects/3d4d78fd087744ecbc5e55e2c4050974/users/53c05b0f094644888f7444f917503a88/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.344647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.345497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.345779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.346057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] PATH_INFO: `/v3/projects/5c80ec96b2f34c42a02d886b574e37c0/users/e5c547a57e5e44fb8a1c89558b480991/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.355791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4d78fd087744ecbc5e55e2c4050974, user_id=53c05b0f094644888f7444f917503a88, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.358555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.360755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c80ec96b2f34c42a02d886b574e37c0, user_id=e5c547a57e5e44fb8a1c89558b480991, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.363443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.371912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 807/1608] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:56 2022] PUT /identity/v3/projects/3d4d78fd087744ecbc5e55e2c4050974/users/53c05b0f094644888f7444f917503a88/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.372735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.374283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.375512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 802/1609] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:56 2022] PUT /identity/v3/projects/5c80ec96b2f34c42a02d886b574e37c0/users/e5c547a57e5e44fb8a1c89558b480991/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.376337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.432003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144aaf-74af-4457-8b2d-9768c8584a53 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.436442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fd7e6-845a-45fd-99d8-6781d7c89f9d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.441076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04c4cb-41a3-4422-bd57-fb44a19b93cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.441749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c4cb-41a3-4422-bd57-fb44a19b93cf admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.442007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c4cb-41a3-4422-bd57-fb44a19b93cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.442278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c4cb-41a3-4422-bd57-fb44a19b93cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.442987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04c4cb-41a3-4422-bd57-fb44a19b93cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.444564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04c4cb-41a3-4422-bd57-fb44a19b93cf admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.453874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 803/1610] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:56.457155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.457155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.457155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.479177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.480076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.480357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.480650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.481931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] MFA Rules not processed for user `8327063b27d34d7f9da9aeae6c4bd198`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.483107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.489693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.491114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.500319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 808/1611] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:56.501815 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.509920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5662d-2adb-4576-ba9a-70bf162af911 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.515693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe0f1-5bcf-4a72-bea2-270d3609114c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.516331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe0f1-5bcf-4a72-bea2-270d3609114c service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.516755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe0f1-5bcf-4a72-bea2-270d3609114c service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.517104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe0f1-5bcf-4a72-bea2-270d3609114c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.529501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abe0f1-5bcf-4a72-bea2-270d3609114c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.531048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abe0f1-5bcf-4a72-bea2-270d3609114c service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.541633 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 809/1612] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:56.542831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3dd58-7493-461f-b88d-91b374410acb None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.543180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3dd58-7493-461f-b88d-91b374410acb None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.543468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3dd58-7493-461f-b88d-91b374410acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.554199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 804/1613] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.554897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.569446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f876a-a59a-4f55-b33a-4ad1b9ccd7bc None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.571446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-26c3dd58-7493-461f-b88d-91b374410acb None None] MFA Rules not processed for user `cbef2598731b4cd9abfd0c8a3bc3cdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.572699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-26c3dd58-7493-461f-b88d-91b374410acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.574883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd7619-69e0-4ed5-bfd6-b205ebc94b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.575856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7619-69e0-4ed5-bfd6-b205ebc94b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.576145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7619-69e0-4ed5-bfd6-b205ebc94b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.576418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7619-69e0-4ed5-bfd6-b205ebc94b6a admin admin] PATH_INFO: `/v3/projects/0d6d170d3db6427f867c5c7caaf295c2/users/867f26290c254777b65cb739263d708a/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.594530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd7619-69e0-4ed5-bfd6-b205ebc94b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6d170d3db6427f867c5c7caaf295c2, user_id=867f26290c254777b65cb739263d708a, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.597260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd7619-69e0-4ed5-bfd6-b205ebc94b6a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.612978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 805/1614] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:56 2022] PUT /identity/v3/projects/0d6d170d3db6427f867c5c7caaf295c2/users/867f26290c254777b65cb739263d708a/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.614620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.614908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.615629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.633442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.634638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.646251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 810/1615] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.647747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.648070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.648359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.670606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.671804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.706446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 806/1616] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.710256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.722118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bde0b-10fb-434f-936b-afbca81a29ac None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.728040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ce58d-50f3-4a65-9894-3b12651ac860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.729223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce58d-50f3-4a65-9894-3b12651ac860 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.729506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce58d-50f3-4a65-9894-3b12651ac860 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.729781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ce58d-50f3-4a65-9894-3b12651ac860 admin admin] PATH_INFO: `/v3/projects/004ab055d0c84fa08d1d238a46c04a3f/users/8fec0b9b99934e90b5c98eb43ca4a448/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.744502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 811/1617] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.744502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.747307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ce58d-50f3-4a65-9894-3b12651ac860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004ab055d0c84fa08d1d238a46c04a3f, user_id=8fec0b9b99934e90b5c98eb43ca4a448, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.750217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ce58d-50f3-4a65-9894-3b12651ac860 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.769874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 807/1618] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:56 2022] PUT /identity/v3/projects/004ab055d0c84fa08d1d238a46c04a3f/users/8fec0b9b99934e90b5c98eb43ca4a448/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:56.776833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.777420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.777714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.801981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:56.803319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:56.819310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e011e-d007-4558-8b81-ab27731dba80 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.873695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 808/1619] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:56.878125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:56.886012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.886674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.887097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.887389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.948779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3a694-37bc-49bf-8637-aa114978f3b1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:56.952976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:56.953124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ffdcd-7403-473b-8196-a2378c5e4351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:56.953800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffdcd-7403-473b-8196-a2378c5e4351 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:56.954077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffdcd-7403-473b-8196-a2378c5e4351 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:56.954401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffdcd-7403-473b-8196-a2378c5e4351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:56.954513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:56.967765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 812/1620] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:56.968460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:57.026803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a113da0-7922-4a03-b576-f8a976545e54 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:57.030911 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:57.031711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:57.032379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:57.032379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:57.051019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:57.052905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:57.066419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 813/1621] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:57.067419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:57.077693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6652fc-3fa8-4829-a53a-d39de6831c4a service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:57.082816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:57.083610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:57.084357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:57.084760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:57.088524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ffdcd-7403-473b-8196-a2378c5e4351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:57.090224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ffdcd-7403-473b-8196-a2378c5e4351 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:57.096617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:57.098500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:57.108443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 809/1622] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:57.112739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 814/1623] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:57.114236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:57.119000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:57.119579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:57.120032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:57.125126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3112e1d-9327-4015-9336-f02cf6b9260b service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:57.130449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:57.131212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:57.131594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:57.131968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:57.143279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:57.144910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:57.147259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:57.148761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:57.156560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 815/1624] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:57.157883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:57.167182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 810/1625] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:57.168395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b883d8-74c6-44e9-8ce0-3febac03ce66 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:57.174938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:57.175759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:57.176262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:57.176670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:57.182458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:57.195323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9247597-4e79-4bca-92f5-1158f3cbd906 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:57.206620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28579c-ffa3-4e2f-9e1c-e27a9d7363e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:57.207292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28579c-ffa3-4e2f-9e1c-e27a9d7363e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:57.208138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28579c-ffa3-4e2f-9e1c-e27a9d7363e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:57.208138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28579c-ffa3-4e2f-9e1c-e27a9d7363e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:57.228176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28579c-ffa3-4e2f-9e1c-e27a9d7363e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:57.229709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28579c-ffa3-4e2f-9e1c-e27a9d7363e7 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:57.250177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:57.252183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:57.254283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 811/1626] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:57.265330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 816/1627] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:50:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:50:58.705056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:58.706035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:58.706957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:58.765134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] MFA Rules not processed for user `167f6e68bc0e4cc9bf551c9ad1430bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:50:58.766782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:50:58.866388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 812/1628] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:50:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:50:58.870965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:58.889625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b10b78-f61b-489a-94ff-77196206a327 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:58.898564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:58.900874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:58.902280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:58.902774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:58.903777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:58.906489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:58.960712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 817/1629] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:50:58 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:58.965518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:58.985403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac0d45-7981-438d-a168-00320e8f2fe7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:59.214833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:59.217860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:59.218148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:59.218466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:59.220443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:59.226800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:59.279617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 813/1630] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:50:58 2022] POST /identity/v3/users => generated 309 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:50:59.284051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:59.300655 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494bfd-0ace-44e5-b4cd-67b38e54c1de admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:59.313683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:59.314364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:59.314644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:59.314920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:59.315658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:59.319461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:59.345001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 818/1631] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:50:59.349466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:59.366218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d57054-17f6-4116-840f-32cd672769bd admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:59.381038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:59.383728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:59.384332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:59.384826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] PATH_INFO: `/v3/projects/5a5b23bff8bd480d8967961777da1499/users/6031a9a4b84045ed8138f332d7f9eb10/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:59.408583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5b23bff8bd480d8967961777da1499, user_id=6031a9a4b84045ed8138f332d7f9eb10, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:59.414933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:59.438127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 814/1632] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:59 2022] PUT /identity/v3/projects/5a5b23bff8bd480d8967961777da1499/users/6031a9a4b84045ed8138f332d7f9eb10/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:50:59.438849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:59.458053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d15ac9-858b-4c71-816f-c51ba5174fd4 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:59.620127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f77b78-80f2-404f-814f-ef4d66da4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:59.620931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77b78-80f2-404f-814f-ef4d66da4613 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:59.622465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77b78-80f2-404f-814f-ef4d66da4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:59.624932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77b78-80f2-404f-814f-ef4d66da4613 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:59.625816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77b78-80f2-404f-814f-ef4d66da4613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:59.627589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77b78-80f2-404f-814f-ef4d66da4613 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:59.666114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 819/1633] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:50:59.672685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:50:59.690259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742c579-b8a6-4583-8d09-df6e14efd69f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:50:59.703506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e27b9-d3f1-4a6e-958e-ad3d00c2d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:50:59.714205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e27b9-d3f1-4a6e-958e-ad3d00c2d3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:50:59.715086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e27b9-d3f1-4a6e-958e-ad3d00c2d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:50:59.715727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e27b9-d3f1-4a6e-958e-ad3d00c2d3a0 admin admin] PATH_INFO: `/v3/projects/5a5b23bff8bd480d8967961777da1499/users/6031a9a4b84045ed8138f332d7f9eb10/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:50:59.742336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e27b9-d3f1-4a6e-958e-ad3d00c2d3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5b23bff8bd480d8967961777da1499, user_id=6031a9a4b84045ed8138f332d7f9eb10, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:50:59.751312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e27b9-d3f1-4a6e-958e-ad3d00c2d3a0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:50:59.795681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 815/1634] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:50:59 2022] PUT /identity/v3/projects/5a5b23bff8bd480d8967961777da1499/users/6031a9a4b84045ed8138f332d7f9eb10/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:00.080822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8114-2030-4ae3-9695-3fcb068fc8f0 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.081760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8114-2030-4ae3-9695-3fcb068fc8f0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.082204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8114-2030-4ae3-9695-3fcb068fc8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.144213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5c1b8114-2030-4ae3-9695-3fcb068fc8f0 None None] MFA Rules not processed for user `53c05b0f094644888f7444f917503a88`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:00.145531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5c1b8114-2030-4ae3-9695-3fcb068fc8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:00.272798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 820/1635] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:00.288944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedcec5-6062-4bc7-ab06-0935437cdf26 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.289832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedcec5-6062-4bc7-ab06-0935437cdf26 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.290095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedcec5-6062-4bc7-ab06-0935437cdf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.312703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1bedcec5-6062-4bc7-ab06-0935437cdf26 None None] MFA Rules not processed for user `53c05b0f094644888f7444f917503a88`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:00.314116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1bedcec5-6062-4bc7-ab06-0935437cdf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:00.327683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 816/1636] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:00.349364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10550d-cc6d-4c1c-8a11-6262c492cb9c None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.349656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10550d-cc6d-4c1c-8a11-6262c492cb9c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.349934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10550d-cc6d-4c1c-8a11-6262c492cb9c None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.351319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 821/1637] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 16 18:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:00.358845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae6f80-d5a6-4806-8cb9-99707fc8361d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.359138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae6f80-d5a6-4806-8cb9-99707fc8361d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.359412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae6f80-d5a6-4806-8cb9-99707fc8361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.399145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c5ae6f80-d5a6-4806-8cb9-99707fc8361d None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:00.399508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c5ae6f80-d5a6-4806-8cb9-99707fc8361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:00.557853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 817/1638] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 16 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:00.562663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10550d-cc6d-4c1c-8a11-6262c492cb9c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:00.595673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10550d-cc6d-4c1c-8a11-6262c492cb9c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:00.608414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbc234-c9ee-4c7c-a933-a6c9fdaad3ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:00.609246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbc234-c9ee-4c7c-a933-a6c9fdaad3ee service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.609540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbc234-c9ee-4c7c-a933-a6c9fdaad3ee service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.609930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbc234-c9ee-4c7c-a933-a6c9fdaad3ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.641547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbc234-c9ee-4c7c-a933-a6c9fdaad3ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:00.643909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbc234-c9ee-4c7c-a933-a6c9fdaad3ee service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:00.686130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 822/1639] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:00.853475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe97a0-3da7-45a2-ad1c-84b48cf4e86b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.854143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe97a0-3da7-45a2-ad1c-84b48cf4e86b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.854599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe97a0-3da7-45a2-ad1c-84b48cf4e86b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.857802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 818/1640] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 16 18:51:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:00.880127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18993ba0-15b1-4726-aa99-9455da7bfdb3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.880467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18993ba0-15b1-4726-aa99-9455da7bfdb3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.880774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18993ba0-15b1-4726-aa99-9455da7bfdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.925986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-18993ba0-15b1-4726-aa99-9455da7bfdb3 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:00.928685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-18993ba0-15b1-4726-aa99-9455da7bfdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:00.979738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 823/1641] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 16 18:51:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:00.989019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5383f99-bbd2-4f2a-bb39-ce92e97152de None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:00.989019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5383f99-bbd2-4f2a-bb39-ce92e97152de None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:00.989268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5383f99-bbd2-4f2a-bb39-ce92e97152de None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:00.991152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 819/1642] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 16 18:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:01.009020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784331c-7ff0-4acc-85e8-2b0388b617e1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:01.009519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784331c-7ff0-4acc-85e8-2b0388b617e1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:01.010028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784331c-7ff0-4acc-85e8-2b0388b617e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:01.046250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c784331c-7ff0-4acc-85e8-2b0388b617e1 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:01.047556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c784331c-7ff0-4acc-85e8-2b0388b617e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:01.072348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 824/1643] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:51:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:01.088305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5383f99-bbd2-4f2a-bb39-ce92e97152de None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:01.144985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f2c6-e277-4b3f-9c52-5d54eac25788 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:01.145826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f2c6-e277-4b3f-9c52-5d54eac25788 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:01.146281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f2c6-e277-4b3f-9c52-5d54eac25788 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:01.148391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 825/1644] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 16 18:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:01.154912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:01.155437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:01.155877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:01.208415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:01.213230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:01.234264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5383f99-bbd2-4f2a-bb39-ce92e97152de None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:01.238961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 826/1645] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:51:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:01.244265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b714563-dd19-4af8-bf02-93644c759e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:01.244793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714563-dd19-4af8-bf02-93644c759e81 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:01.245081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714563-dd19-4af8-bf02-93644c759e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:01.245356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714563-dd19-4af8-bf02-93644c759e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:01.252419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:01.254291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:01.278881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b714563-dd19-4af8-bf02-93644c759e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:01.280419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b714563-dd19-4af8-bf02-93644c759e81 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:01.317496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 820/1646] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:01.324584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920acd7-01ee-40b5-bc23-b7d93c1ce150 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:01.337034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82893a-0626-4ca9-a1c1-ea1c88527016 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:01.337709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82893a-0626-4ca9-a1c1-ea1c88527016 service glance] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:01.338056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82893a-0626-4ca9-a1c1-ea1c88527016 service glance] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:01.338331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82893a-0626-4ca9-a1c1-ea1c88527016 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:01.362914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82893a-0626-4ca9-a1c1-ea1c88527016 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:01.367864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82893a-0626-4ca9-a1c1-ea1c88527016 service glance] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:01.392524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 827/1647] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:02.004861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9b38-605c-4318-a87d-ece31745d2c8 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:02.006022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9b38-605c-4318-a87d-ece31745d2c8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:02.006502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff9b38-605c-4318-a87d-ece31745d2c8 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:02.009463 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 821/1648] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 16 18:51:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:02.018786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648b8fd-180a-4eb0-96ff-65a43fe33f71 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:02.018786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648b8fd-180a-4eb0-96ff-65a43fe33f71 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:02.019089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648b8fd-180a-4eb0-96ff-65a43fe33f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:02.072767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-3648b8fd-180a-4eb0-96ff-65a43fe33f71 None None] MFA Rules not processed for user `e653231605f345b0846ac42508e7d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:02.074422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3648b8fd-180a-4eb0-96ff-65a43fe33f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:02.138556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 828/1649] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:51:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:02.158829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff9b38-605c-4318-a87d-ece31745d2c8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:02.175909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff9b38-605c-4318-a87d-ece31745d2c8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:02.188149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800c537-4186-4e9c-aeec-a663cce9e98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:02.189237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800c537-4186-4e9c-aeec-a663cce9e98c service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:02.189867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800c537-4186-4e9c-aeec-a663cce9e98c service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:02.190350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800c537-4186-4e9c-aeec-a663cce9e98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:02.200569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800c537-4186-4e9c-aeec-a663cce9e98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:02.204189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800c537-4186-4e9c-aeec-a663cce9e98c service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:02.218662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 822/1650] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:02.620895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ba600-ee85-4ddd-b135-12a1f66eca45 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:02.621422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ba600-ee85-4ddd-b135-12a1f66eca45 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:02.621422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ba600-ee85-4ddd-b135-12a1f66eca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:02.673973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-276ba600-ee85-4ddd-b135-12a1f66eca45 None None] MFA Rules not processed for user `6031a9a4b84045ed8138f332d7f9eb10`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:02.675263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-276ba600-ee85-4ddd-b135-12a1f66eca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:02.807976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 829/1651] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:02 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:03.531008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a84460-f4bd-4830-b83e-6cee62ae1eb2 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:03.531510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a84460-f4bd-4830-b83e-6cee62ae1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:03.531609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a84460-f4bd-4830-b83e-6cee62ae1eb2 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:03.533264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 823/1652] 10.222.0.36 () {54 vars in 960 bytes} [Mon May 16 18:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:03.540502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:03.541340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:03.542224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:03.566536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] MFA Rules not processed for user `c103464c1bca4da29e61fe73b6fd1d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:03.568041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:03.678036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 830/1653] 10.222.0.36 () {58 vars in 1057 bytes} [Mon May 16 18:51:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:03.688517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a84460-f4bd-4830-b83e-6cee62ae1eb2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:03.708469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a84460-f4bd-4830-b83e-6cee62ae1eb2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:03.725679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:03.726337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:03.727150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:03.727425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:03.729318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:03.758125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:03.760030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:03.778509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 824/1654] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:06.159889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:06.318124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582a341-65f6-47ec-81bf-b209d891ad86 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:06.328396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:06.329706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:06.330184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:06.330649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] PATH_INFO: `/v3/users/167f6e68bc0e4cc9bf551c9ad1430bb7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:06.332167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] RBAC: Authorizing `identity:delete_user(user_id=167f6e68bc0e4cc9bf551c9ad1430bb7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:06.335253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:06.437804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167f6e68bc0e4cc9bf551c9ad1430bb7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:06.447179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167f6e68bc0e4cc9bf551c9ad1430bb7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:06.448339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167f6e68bc0e4cc9bf551c9ad1430bb7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:06.463672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167f6e68bc0e4cc9bf551c9ad1430bb7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:06.464613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167f6e68bc0e4cc9bf551c9ad1430bb7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:06.518221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 831/1655] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:06 2022] DELETE /identity/v3/users/167f6e68bc0e4cc9bf551c9ad1430bb7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:06.900314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:06.919334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b6d2a-3b2e-43e8-a16f-1348a02e21ae service placement] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:07.120773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:07.124549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:07.124752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:07.124897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] PATH_INFO: `/v3/projects/96b041b622fb4e0fa52ae670ba7b9505` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:07.133969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b041b622fb4e0fa52ae670ba7b9505)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:07.135852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:07.245811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b041b622fb4e0fa52ae670ba7b9505', 'request_id': 'req-6fc4605b-6535-4015-9577-aaa6e85c9c8a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.278611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b041b622fb4e0fa52ae670ba7b9505', 'request_id': 'req-6fc4605b-6535-4015-9577-aaa6e85c9c8a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.317972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b041b622fb4e0fa52ae670ba7b9505', 'request_id': 'req-6fc4605b-6535-4015-9577-aaa6e85c9c8a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.350805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] The token cache is being invalidate because project 96b041b622fb4e0fa52ae670ba7b9505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:07.351092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.351799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.353647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 825/1656] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:06 2022] DELETE /identity/v3/projects/96b041b622fb4e0fa52ae670ba7b9505 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:07.357505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:07.359485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:07.391271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e051-0797-4d68-b682-6359d38f633c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:07.467962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:07.469143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:07.469562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:07.469973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] PATH_INFO: `/v3/users/6031a9a4b84045ed8138f332d7f9eb10` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:07.471372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6031a9a4b84045ed8138f332d7f9eb10)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:07.474547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:07.540171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6031a9a4b84045ed8138f332d7f9eb10'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.551454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6031a9a4b84045ed8138f332d7f9eb10'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.551454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6031a9a4b84045ed8138f332d7f9eb10'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.568708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6031a9a4b84045ed8138f332d7f9eb10'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.569672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f1af99fd-de89-4fe4-bfa8-c2d9bb62e06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6031a9a4b84045ed8138f332d7f9eb10'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:07.604678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 832/1657] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:07 2022] DELETE /identity/v3/users/6031a9a4b84045ed8138f332d7f9eb10 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:08.039583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:08.045156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:08.066590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4605b-6535-4015-9577-aaa6e85c9c8a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:08.143443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:08.144500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:08.145941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:08.146666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] PATH_INFO: `/v3/projects/5a5b23bff8bd480d8967961777da1499` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:08.149319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5b23bff8bd480d8967961777da1499)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:08.153214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:08.260973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5b23bff8bd480d8967961777da1499', 'request_id': 'req-71aba51e-f562-47a6-af1f-1d7ea5bc1962'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:08.269500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5b23bff8bd480d8967961777da1499', 'request_id': 'req-71aba51e-f562-47a6-af1f-1d7ea5bc1962'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:08.285621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5b23bff8bd480d8967961777da1499', 'request_id': 'req-71aba51e-f562-47a6-af1f-1d7ea5bc1962'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:08.302575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] The token cache is being invalidate because project 5a5b23bff8bd480d8967961777da1499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:08.303286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:08.304208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71aba51e-f562-47a6-af1f-1d7ea5bc1962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:08.308341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 826/1658] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:08 2022] DELETE /identity/v3/projects/5a5b23bff8bd480d8967961777da1499 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:08.314100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:08.314640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:08.315131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:08.344576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:08.346290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:08.553968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 833/1659] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:08.562118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898c394-028a-4c11-9626-7861716e71ff None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:08.562432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898c394-028a-4c11-9626-7861716e71ff None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:08.562722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898c394-028a-4c11-9626-7861716e71ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:08.608360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4898c394-028a-4c11-9626-7861716e71ff None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:08.614830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4898c394-028a-4c11-9626-7861716e71ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:08.643470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 827/1660] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:08.647438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:08.667063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90be791-1eb6-4b83-98f5-f297b7b2c47c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:08.685357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:08.686410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:08.686896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:08.687450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:08.688392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:08.689091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:08.690933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:08.737990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 834/1661] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:08.738704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898c394-028a-4c11-9626-7861716e71ff None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:08.749359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898c394-028a-4c11-9626-7861716e71ff None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:08.766109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08484cb4-e64f-4ae8-91f0-1885725f2793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:08.766794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08484cb4-e64f-4ae8-91f0-1885725f2793 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:08.767078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08484cb4-e64f-4ae8-91f0-1885725f2793 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:08.767352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08484cb4-e64f-4ae8-91f0-1885725f2793 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:08.768056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08484cb4-e64f-4ae8-91f0-1885725f2793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:08.774660 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08484cb4-e64f-4ae8-91f0-1885725f2793 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:08.857700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 828/1662] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:08 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:08.861812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:08.883464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77236-01df-4b75-bcff-82b5300624b8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:09.020084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:09.022006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.024217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.024721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.029422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:09.031701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:09.040471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.041933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.043056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.090990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:09.096177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:09.108517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 835/1663] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:08 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:09.115578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:09.135805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264322fc-eafe-4df6-9e17-e60993fb3322 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:09.156311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:09.156961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.157247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.157521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.158251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:09.181255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:09.209706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 836/1664] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:09.211507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:09.241347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd25e9-8e89-4731-9913-46d30a28dd37 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:09.282904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7db28-1f88-4d22-9561-a87b924afe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:09.288030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7db28-1f88-4d22-9561-a87b924afe0f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.289525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7db28-1f88-4d22-9561-a87b924afe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.290579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7db28-1f88-4d22-9561-a87b924afe0f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.293773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 829/1665] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:09.299184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7db28-1f88-4d22-9561-a87b924afe0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:09.303899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7db28-1f88-4d22-9561-a87b924afe0f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:09.339672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 837/1666] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:09.340871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:09.362654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b8896-c848-4c6e-9304-d231b0090a8e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:09.391007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9f28c-bb3f-4616-9a4f-bb69ad938b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:09.396257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9f28c-bb3f-4616-9a4f-bb69ad938b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.396257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9f28c-bb3f-4616-9a4f-bb69ad938b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.396257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9f28c-bb3f-4616-9a4f-bb69ad938b56 admin admin] PATH_INFO: `/v3/projects/15faa2e2910d4dadbde1dca31897812c/users/f0a89a541ee34ee08d1971550e946a0a/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.422030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9f28c-bb3f-4616-9a4f-bb69ad938b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15faa2e2910d4dadbde1dca31897812c, user_id=f0a89a541ee34ee08d1971550e946a0a, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:09.425400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9f28c-bb3f-4616-9a4f-bb69ad938b56 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:09.467416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 830/1667] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:09 2022] PUT /identity/v3/projects/15faa2e2910d4dadbde1dca31897812c/users/f0a89a541ee34ee08d1971550e946a0a/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:09.477771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.478059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.478498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.529995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.532559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.533167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:09.546947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] MFA Rules not processed for user `f0a89a541ee34ee08d1971550e946a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:09.548206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:09.650105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] MFA Rules not processed for user `3b39f00235c34612b591cd2d974d9969`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:09.651968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:09.803529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 838/1668] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:09.805597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:09.807293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:09.855616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 831/1669] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:09.859133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:09.961126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734f84b-6c2d-401b-8574-c3a7bfa5b9c6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:09.969325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d31779-5ea6-4864-a29c-f1b88def1dc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:09.970339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d31779-5ea6-4864-a29c-f1b88def1dc2 service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:09.970652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d31779-5ea6-4864-a29c-f1b88def1dc2 service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:09.970887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d31779-5ea6-4864-a29c-f1b88def1dc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.092534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b087-b388-4e31-9958-08a05ed8f04f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:10.106540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d31779-5ea6-4864-a29c-f1b88def1dc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:10.111352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d31779-5ea6-4864-a29c-f1b88def1dc2 service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:10.115279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:10.115968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:10.116252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:10.116530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.117883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:10.119870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:10.137503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 839/1670] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:51:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:51:10.144739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:10.145045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:10.145415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.194462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] MFA Rules not processed for user `3b39f00235c34612b591cd2d974d9969`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:10.200051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:10.204303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 832/1671] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:09 2022] POST /identity/v3/projects => generated 379 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:10.215155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:10.347606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f5b9-079c-46c3-90a3-537028147077 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:10.367883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:10.368579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:10.368895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:10.369168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.369876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:10.371853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:10.437115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 833/1672] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:10 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:10.441010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:10.477252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04d438-b577-459b-9cec-c41a326f5425 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:10.492246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 840/1673] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:10.502240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:10.503792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:10.504105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:10.504967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:10.505491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:10.505924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.506879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:10.511975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:10.537995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 834/1674] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:10.542080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:10.563105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962373ad-8294-4b6d-b456-4f4eedf5bc92 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:10.570988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:10.572098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:10.572515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:10.572952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] PATH_INFO: `/v3/projects/4126e23dd09a4fe2a724fa1461071d9d/users/a4be4be48e404e9e82ef581ac4cc5129/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.605864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4126e23dd09a4fe2a724fa1461071d9d, user_id=a4be4be48e404e9e82ef581ac4cc5129, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:10.609579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:10.635880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 835/1675] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:10 2022] PUT /identity/v3/projects/4126e23dd09a4fe2a724fa1461071d9d/users/a4be4be48e404e9e82ef581ac4cc5129/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:10.640235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:10.813743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d10b8-4d85-4019-8a21-3672d5fdf80f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:10.871182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39de13-8e91-4e21-b509-bd55e461dcff admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:10.889008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:10.889697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:10.889977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:10.890344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:10.891037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:10.900968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:10.938189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 836/1676] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:10.946143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:10.985200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3310-6f8c-4008-9248-e27ffb531461 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:11.007158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890b005-b425-479a-b1c3-e2491ea28952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:11.008194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890b005-b425-479a-b1c3-e2491ea28952 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:11.008384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890b005-b425-479a-b1c3-e2491ea28952 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:11.008689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890b005-b425-479a-b1c3-e2491ea28952 admin admin] PATH_INFO: `/v3/projects/4126e23dd09a4fe2a724fa1461071d9d/users/a4be4be48e404e9e82ef581ac4cc5129/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:11.027668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890b005-b425-479a-b1c3-e2491ea28952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4126e23dd09a4fe2a724fa1461071d9d, user_id=a4be4be48e404e9e82ef581ac4cc5129, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:11.033145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890b005-b425-479a-b1c3-e2491ea28952 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:11.079578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:11.080378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:11.080834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:11.081439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:11.093818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 837/1677] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:10 2022] PUT /identity/v3/projects/4126e23dd09a4fe2a724fa1461071d9d/users/a4be4be48e404e9e82ef581ac4cc5129/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:11.106179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:11.106682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:11.107128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:11.211175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] MFA Rules not processed for user `a4be4be48e404e9e82ef581ac4cc5129`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:11.212516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:11.263188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:11.267738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:11.327046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 841/1678] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 825 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:11.460994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 838/1679] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:11.474127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:11.475734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:11.701940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467172f-88c2-4c2a-8411-d59db98ff70a service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:11.886036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eb14b-863e-45b2-8017-f4c464ae29a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:11.886701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb14b-863e-45b2-8017-f4c464ae29a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:11.887151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb14b-863e-45b2-8017-f4c464ae29a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:11.887577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb14b-863e-45b2-8017-f4c464ae29a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:11.932097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eb14b-863e-45b2-8017-f4c464ae29a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:11.934181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eb14b-863e-45b2-8017-f4c464ae29a0 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:11.984078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 842/1680] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:12.333761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:12.361858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aba85-93ca-4ffa-843e-a51d0fe53ee6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:12.371689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:12.372447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:12.372846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:12.373349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:12.432593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cd9a7-8d8d-41f5-93bc-373128bdb031 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:12.433003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cd9a7-8d8d-41f5-93bc-373128bdb031 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:12.433327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cd9a7-8d8d-41f5-93bc-373128bdb031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:12.489017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f03cd9a7-8d8d-41f5-93bc-373128bdb031 None None] MFA Rules not processed for user `e653231605f345b0846ac42508e7d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:12.490284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f03cd9a7-8d8d-41f5-93bc-373128bdb031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:12.582460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:12.589294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:12.614623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 839/1681] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:12.670934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 843/1682] 10.222.0.59 () {58 vars in 1015 bytes} [Mon May 16 18:51:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:12.694008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:12.695874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:12.715923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174d609-c622-4ecd-a231-5fe36c6b962a service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:12.737132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea26713-6131-4ba4-8e70-bbc3ace0614e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:12.739527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea26713-6131-4ba4-8e70-bbc3ace0614e service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:12.740895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea26713-6131-4ba4-8e70-bbc3ace0614e service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:12.741405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea26713-6131-4ba4-8e70-bbc3ace0614e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:12.762644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea26713-6131-4ba4-8e70-bbc3ace0614e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:12.770539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea26713-6131-4ba4-8e70-bbc3ace0614e service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:12.803320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 840/1683] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:15.334164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1fac-96e7-4a0e-b265-60bf1f5293e6 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:15.334712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1fac-96e7-4a0e-b265-60bf1f5293e6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:15.334712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1fac-96e7-4a0e-b265-60bf1f5293e6 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:15.336081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 844/1684] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 16 18:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:51:15.344153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634de186-de36-455f-aaed-169a887d9ab2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:15.344699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634de186-de36-455f-aaed-169a887d9ab2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:15.344926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634de186-de36-455f-aaed-169a887d9ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:15.433268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-634de186-de36-455f-aaed-169a887d9ab2 None None] MFA Rules not processed for user `e653231605f345b0846ac42508e7d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:15.436341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-634de186-de36-455f-aaed-169a887d9ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:15.486316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 841/1685] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:51:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:15.513313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda406e-1509-4be3-86e4-86c7a004f583 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:15.515263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda406e-1509-4be3-86e4-86c7a004f583 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:15.515881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda406e-1509-4be3-86e4-86c7a004f583 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:15.517831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 845/1686] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:51:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:15.545868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a1847-540e-45b1-859b-d81ac67d8e91 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:15.546159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a1847-540e-45b1-859b-d81ac67d8e91 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:15.546491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a1847-540e-45b1-859b-d81ac67d8e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:15.623241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-db9a1847-540e-45b1-859b-d81ac67d8e91 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:15.628606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda406e-1509-4be3-86e4-86c7a004f583 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:15.630366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-db9a1847-540e-45b1-859b-d81ac67d8e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:15.633750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0dda406e-1509-4be3-86e4-86c7a004f583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:15.696234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda406e-1509-4be3-86e4-86c7a004f583 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:15.711042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:15.711717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:15.711996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:15.712267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:15.771835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:15.774851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:15.810963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 846/1687] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:15.814839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 842/1688] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:51:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:15.876112 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:15.878354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:15.940898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91fec9-43d8-449d-a4b2-83d261e18294 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:15.952895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817464-ce93-4bb4-8d2d-1d5b2c26e82b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:15.953711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817464-ce93-4bb4-8d2d-1d5b2c26e82b service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:15.954089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817464-ce93-4bb4-8d2d-1d5b2c26e82b service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:15.954565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817464-ce93-4bb4-8d2d-1d5b2c26e82b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:15.967885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817464-ce93-4bb4-8d2d-1d5b2c26e82b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:15.969976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817464-ce93-4bb4-8d2d-1d5b2c26e82b service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:15.986334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 847/1689] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:17.707528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:17.709645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:17.711103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:17.750324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] MFA Rules not processed for user `3b39f00235c34612b591cd2d974d9969`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:17.756425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:18.061305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 843/1690] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:18.838403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd4a45-f56b-4356-ba4c-77881ecf177d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:18.839722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd4a45-f56b-4356-ba4c-77881ecf177d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:18.840564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd4a45-f56b-4356-ba4c-77881ecf177d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:18.898576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-26dd4a45-f56b-4356-ba4c-77881ecf177d None None] MFA Rules not processed for user `e5c547a57e5e44fb8a1c89558b480991`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:18.902123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-26dd4a45-f56b-4356-ba4c-77881ecf177d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:19.035815 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 848/1691] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:19.051737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:19.071795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cafe2-0e42-4cdf-8d40-0f2ca4a111d7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:19.078723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:19.079465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:19.079838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:19.080169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:19.092924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:19.095059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:19.117340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 844/1692] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:19.209203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b836d-da31-4e7a-85e5-ddae8d75a560 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:19.211288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b836d-da31-4e7a-85e5-ddae8d75a560 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:19.211752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b836d-da31-4e7a-85e5-ddae8d75a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:19.285914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-299b836d-da31-4e7a-85e5-ddae8d75a560 None None] MFA Rules not processed for user `8fec0b9b99934e90b5c98eb43ca4a448`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:19.287253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-299b836d-da31-4e7a-85e5-ddae8d75a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:19.510509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 849/1693] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:19.875223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:19.908251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a06cb-ec7a-4279-937b-60bdfd3e5fe8 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:19.927695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:19.929547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:19.929978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:19.930415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:19.958976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:19.969898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:20.016397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 845/1694] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:20.977895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:20.978784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:20.979210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:21.045113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] MFA Rules not processed for user `867f26290c254777b65cb739263d708a`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:21.046650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:21.208675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 850/1695] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:21.229216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:21.243488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93434ee-3a95-47d6-8aa0-554ee4cd321a service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:21.259727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a53231-11ed-47db-b349-c8af71056832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:21.260370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a53231-11ed-47db-b349-c8af71056832 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:21.260899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a53231-11ed-47db-b349-c8af71056832 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:21.261337 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a53231-11ed-47db-b349-c8af71056832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:21.289763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a53231-11ed-47db-b349-c8af71056832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:21.292025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a53231-11ed-47db-b349-c8af71056832 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:21.310978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 846/1696] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:24.070301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:24.071900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:24.386526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2098f4-fe83-4aa7-a71d-c36a58fbb68a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:24.402916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:24.405807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:24.408975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:24.409589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] PATH_INFO: `/v3/users/cbef2598731b4cd9abfd0c8a3bc3cdbe` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:24.413604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbef2598731b4cd9abfd0c8a3bc3cdbe)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:24.418527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:24.581691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbef2598731b4cd9abfd0c8a3bc3cdbe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:24.609620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbef2598731b4cd9abfd0c8a3bc3cdbe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:24.610388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbef2598731b4cd9abfd0c8a3bc3cdbe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:24.635538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbef2598731b4cd9abfd0c8a3bc3cdbe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:24.636279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbef2598731b4cd9abfd0c8a3bc3cdbe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:24.764320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 851/1697] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:24 2022] DELETE /identity/v3/users/cbef2598731b4cd9abfd0c8a3bc3cdbe => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:24.766858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:24.768068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:24.768681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:24.796143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:24.800349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:24.921609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 847/1698] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:25.029027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:25.030659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:25.192339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde396b-812d-4950-b7b8-f64efeb59e47 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:25.206040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f713-1a1e-4039-a710-ccb1097cf1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:25.207389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f713-1a1e-4039-a710-ccb1097cf1bf service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:25.207947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f713-1a1e-4039-a710-ccb1097cf1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:25.208521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f713-1a1e-4039-a710-ccb1097cf1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:25.243245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f713-1a1e-4039-a710-ccb1097cf1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:25.247447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f713-1a1e-4039-a710-ccb1097cf1bf service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:25.279090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 852/1699] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:26.208840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:26.210406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:26.240078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f0ff-ffc1-4a6d-a5df-3cd78e0a1347 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:26.260969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:26.262043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:26.262476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:26.262750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] PATH_INFO: `/v3/users/f0a89a541ee34ee08d1971550e946a0a` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:26.266468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a89a541ee34ee08d1971550e946a0a)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:26.267514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: WARNING py.warnings [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:51:26.267514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: warnings.warn(msg) May 16 18:51:26.267514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: May 16 18:51:26.275041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:26.458746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a89a541ee34ee08d1971550e946a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:26.463536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a89a541ee34ee08d1971550e946a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:26.513266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a89a541ee34ee08d1971550e946a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:26.553638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a89a541ee34ee08d1971550e946a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:26.556889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a89a541ee34ee08d1971550e946a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:26.653306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 848/1700] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:26 2022] DELETE /identity/v3/users/f0a89a541ee34ee08d1971550e946a0a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:26.657732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:26.658654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:26.659082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:26.706701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:26.710692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:26.893559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 853/1701] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:27.316601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:27.318904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:27.484497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a7a44-197c-4bce-93b8-cf18bfe966ad admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:27.500302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673f7c-c8da-4fd2-9149-db16ef6b6935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:27.503922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673f7c-c8da-4fd2-9149-db16ef6b6935 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:27.504265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673f7c-c8da-4fd2-9149-db16ef6b6935 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:27.504587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673f7c-c8da-4fd2-9149-db16ef6b6935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:27.531530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03673f7c-c8da-4fd2-9149-db16ef6b6935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:27.535614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03673f7c-c8da-4fd2-9149-db16ef6b6935 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:27.567450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 849/1702] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:28.094983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:28.097797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:28.115027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6994-0ab4-4aad-9aff-fd0aa2b743d5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.115594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6994-0ab4-4aad-9aff-fd0aa2b743d5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.116261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d6994-0ab4-4aad-9aff-fd0aa2b743d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.156780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b54bf2-41cd-46f4-8f0c-6375331fa5aa None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:28.163082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:28.164511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.165045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.165476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] PATH_INFO: `/v3/projects/69a6740a82ca4787a113f960eb794573` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.167833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a6740a82ca4787a113f960eb794573)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:28.169017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-431d6994-0ab4-4aad-9aff-fd0aa2b743d5 None None] MFA Rules not processed for user `e5c547a57e5e44fb8a1c89558b480991`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:28.169768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: WARNING py.warnings [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 16 18:51:28.169768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: warnings.warn(msg) May 16 18:51:28.169768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: May 16 18:51:28.170251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-431d6994-0ab4-4aad-9aff-fd0aa2b743d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:28.172248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:28.299986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a6740a82ca4787a113f960eb794573', 'request_id': 'req-199090eb-d837-4730-8987-1ff60d30fcb5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:28.308399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a6740a82ca4787a113f960eb794573', 'request_id': 'req-199090eb-d837-4730-8987-1ff60d30fcb5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:28.336205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a6740a82ca4787a113f960eb794573', 'request_id': 'req-199090eb-d837-4730-8987-1ff60d30fcb5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:28.346432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 850/1703] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:28.363786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] The token cache is being invalidate because project 69a6740a82ca4787a113f960eb794573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:28.364285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:28.366401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-199090eb-d837-4730-8987-1ff60d30fcb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:28.369795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 854/1704] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:28 2022] DELETE /identity/v3/projects/69a6740a82ca4787a113f960eb794573 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:28.374784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de2471-0326-4b72-a540-21552122a32f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.375076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de2471-0326-4b72-a540-21552122a32f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.375451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de2471-0326-4b72-a540-21552122a32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.401805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-77de2471-0326-4b72-a540-21552122a32f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:28.403406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-77de2471-0326-4b72-a540-21552122a32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:28.509743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.510225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.510731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.548038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] MFA Rules not processed for user `8fec0b9b99934e90b5c98eb43ca4a448`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:28.549603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:28.572231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 851/1705] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:28.581127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.581400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.581683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.619927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:28.622136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:28.634847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 852/1706] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:28.635937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:28.641868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 855/1707] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:28.652759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6321-5ad3-426f-b506-21883e0775ef None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:28.658181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:28.659213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.660056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.660400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.661306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:28.661684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:28.666514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:28.679228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 853/1708] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:28.683530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:28.694881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e104b-1eed-4398-b835-31ce2f57fd9c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:28.700002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:28.700834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:28.701115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:28.701386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:28.702067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:28.704002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:28.758369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 856/1709] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:28 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:28.759795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:28.772118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6498aa9-ba6c-4102-9179-82639f57b328 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:28.984516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:28.986743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:29.026352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.027742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.028165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.028589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.030328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.042050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.073537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 854/1710] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:28 2022] POST /identity/v3/users => generated 286 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:29.079135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.091536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd7362-ca84-40c8-a0ea-10b9d234b3c4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.096780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.097642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.098084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.098507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.099364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.101665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.111409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09972e-1d94-4c9b-8cd6-629a67cdb009 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.116724 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 855/1711] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:29.120590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.134554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328e355-ad1d-4a11-a786-43091e8c34ff admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.144934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.145760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.145900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.146146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.146591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] PATH_INFO: `/v3/projects/15faa2e2910d4dadbde1dca31897812c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.146700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.147129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.147469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.148392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.150214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15faa2e2910d4dadbde1dca31897812c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.152288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.153113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.168848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 856/1712] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:29.175800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.196319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978907d-c13c-473e-84c8-48b5ef92aecb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.205076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730d2ee-4f9d-4a6e-86a8-c107481b3fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.206162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d2ee-4f9d-4a6e-86a8-c107481b3fac admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.206603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d2ee-4f9d-4a6e-86a8-c107481b3fac admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.207141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d2ee-4f9d-4a6e-86a8-c107481b3fac admin admin] PATH_INFO: `/v3/projects/788aaad0eeda440fa14a3acfa8ec52f2/users/b647ab0681764239a5f003ad054de1b5/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.251741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730d2ee-4f9d-4a6e-86a8-c107481b3fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=788aaad0eeda440fa14a3acfa8ec52f2, user_id=b647ab0681764239a5f003ad054de1b5, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.254740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730d2ee-4f9d-4a6e-86a8-c107481b3fac admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.264368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15faa2e2910d4dadbde1dca31897812c', 'request_id': 'req-c9789bf5-4174-4b58-896f-90e5d3b02537'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.264368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15faa2e2910d4dadbde1dca31897812c', 'request_id': 'req-c9789bf5-4174-4b58-896f-90e5d3b02537'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.287681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15faa2e2910d4dadbde1dca31897812c', 'request_id': 'req-c9789bf5-4174-4b58-896f-90e5d3b02537'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.298472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 857/1713] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:29 2022] PUT /identity/v3/projects/788aaad0eeda440fa14a3acfa8ec52f2/users/b647ab0681764239a5f003ad054de1b5/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:29.298472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.298472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.298472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.314768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] The token cache is being invalidate because project 15faa2e2910d4dadbde1dca31897812c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:29.315187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.316932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c9789bf5-4174-4b58-896f-90e5d3b02537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.321291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 857/1714] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:28 2022] DELETE /identity/v3/projects/15faa2e2910d4dadbde1dca31897812c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:29.323068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.326656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.327229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.336704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] MFA Rules not processed for user `867f26290c254777b65cb739263d708a`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:29.341702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:29.385790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] MFA Rules not processed for user `b647ab0681764239a5f003ad054de1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:29.390540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:29.453777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 858/1715] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:29.453777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.453777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:29.502549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 858/1716] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:29.509454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.511412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:29.590705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b062237-c831-4b91-8f7b-585f7121c093 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.598401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26884053-7f6a-4c8c-92a4-f25d20132df8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.601974 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.602215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.602908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.602908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.603434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.605796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.613420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.614295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.614568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.614833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] PATH_INFO: `/v3/users/a4be4be48e404e9e82ef581ac4cc5129` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.616394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4be4be48e404e9e82ef581ac4cc5129)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.618349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.643732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 859/1717] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:29 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:29.648605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.712968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4be4be48e404e9e82ef581ac4cc5129'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.714073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4be4be48e404e9e82ef581ac4cc5129'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.732533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4be4be48e404e9e82ef581ac4cc5129'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.750973 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4be4be48e404e9e82ef581ac4cc5129'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.770907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4be4be48e404e9e82ef581ac4cc5129'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:29.804079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26431-53d7-40e2-8805-3b47f0107609 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.818127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.819796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.819796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.819991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.820560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.822500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:29.823901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 859/1718] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:29 2022] DELETE /identity/v3/users/a4be4be48e404e9e82ef581ac4cc5129 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:29.866108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 860/1719] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:29 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:29.870003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:29.871834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:29.977911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a3ac2-ad50-4ead-b0fa-93d1bd0994a9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:29.983673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:29.984499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:29.984916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:29.985393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:29.986300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:29.988206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:30.026873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 860/1720] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:30.028362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:30.042188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca494f-1a0f-46c1-abce-b41d73a1415b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:30.078567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:30.079492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:30.079773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:30.080051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] PATH_INFO: `/v3/projects/4801f8314157432ca4f4d95c86d66177/users/7cc470dcca1742c2ae5b366a709a218e/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:30.123731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4801f8314157432ca4f4d95c86d66177, user_id=7cc470dcca1742c2ae5b366a709a218e, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:30.133539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:30.155643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 861/1721] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:30 2022] PUT /identity/v3/projects/4801f8314157432ca4f4d95c86d66177/users/7cc470dcca1742c2ae5b366a709a218e/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:30.159810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:30.174418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125448ad-5407-44f1-9dc6-cbaf9b13e037 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:30.324335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564fdfa-2f12-4756-9f4f-1c332dd0600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:30.325262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564fdfa-2f12-4756-9f4f-1c332dd0600c admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:30.325808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564fdfa-2f12-4756-9f4f-1c332dd0600c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:30.326190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564fdfa-2f12-4756-9f4f-1c332dd0600c admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:30.327050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564fdfa-2f12-4756-9f4f-1c332dd0600c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:30.329067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564fdfa-2f12-4756-9f4f-1c332dd0600c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:30.345142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 861/1722] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:30.346389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:30.366512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c9eb-819c-4d61-b04a-b0210c942b94 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:30.377373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:30.378549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:30.378910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:30.379258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] PATH_INFO: `/v3/projects/4801f8314157432ca4f4d95c86d66177/users/7cc470dcca1742c2ae5b366a709a218e/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:30.385271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4801f8314157432ca4f4d95c86d66177, user_id=7cc470dcca1742c2ae5b366a709a218e, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:30.388221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:30.431345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 862/1723] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:30 2022] PUT /identity/v3/projects/4801f8314157432ca4f4d95c86d66177/users/7cc470dcca1742c2ae5b366a709a218e/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:30.442618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4b80e-0df8-4239-ad73-aca222fcad52 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:30.443631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4b80e-0df8-4239-ad73-aca222fcad52 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:30.444113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4b80e-0df8-4239-ad73-aca222fcad52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:30.537355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7df4b80e-0df8-4239-ad73-aca222fcad52 None None] MFA Rules not processed for user `7cc470dcca1742c2ae5b366a709a218e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:30.540598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7df4b80e-0df8-4239-ad73-aca222fcad52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:30.685090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 862/1724] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:30.696808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:30.700310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:30.817546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590a625-cf2d-43be-be3e-92d879c47a7e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:30.821938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab81d40-f18e-4e7d-940f-034fe71acfb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:30.822494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81d40-f18e-4e7d-940f-034fe71acfb1 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:30.822779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81d40-f18e-4e7d-940f-034fe71acfb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:30.823108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81d40-f18e-4e7d-940f-034fe71acfb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:30.836870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab81d40-f18e-4e7d-940f-034fe71acfb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:30.838436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab81d40-f18e-4e7d-940f-034fe71acfb1 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:30.862410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 863/1725] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:31.263977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eda9b7-3826-4767-bee6-b6a14fa3a3d3 None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:31.264267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eda9b7-3826-4767-bee6-b6a14fa3a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:31.264548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eda9b7-3826-4767-bee6-b6a14fa3a3d3 None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:31.266362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 863/1726] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:51:31.293427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:31.293943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:31.294118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:31.363278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:31.365536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:31.660191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 864/1727] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:51:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:31.870701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eda9b7-3826-4767-bee6-b6a14fa3a3d3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:31.873636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-13eda9b7-3826-4767-bee6-b6a14fa3a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:31.893387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:31.895443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:31.956126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eda9b7-3826-4767-bee6-b6a14fa3a3d3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:31.976691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:31.977507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:31.977791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:31.978433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.007351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.011787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.051848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 864/1728] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:32.058145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:32.081323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529a92a-4f08-420b-9423-b8d9de59b0f2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.087667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.088656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.088948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.089299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] PATH_INFO: `/v3/projects/4126e23dd09a4fe2a724fa1461071d9d` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.090681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4126e23dd09a4fe2a724fa1461071d9d)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.093172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.099282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acbb33-47b1-480e-9b65-c803698855bf service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.114765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.118773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.119255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.121513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] PATH_INFO: `/v3/users/b647ab0681764239a5f003ad054de1b5` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.125609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b647ab0681764239a5f003ad054de1b5)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.132108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.220933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b647ab0681764239a5f003ad054de1b5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.221854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b647ab0681764239a5f003ad054de1b5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.229531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4126e23dd09a4fe2a724fa1461071d9d', 'request_id': 'req-18fb336c-d2e4-477a-97c4-5e335be50ba4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.235648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b647ab0681764239a5f003ad054de1b5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.246499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b647ab0681764239a5f003ad054de1b5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.246951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4126e23dd09a4fe2a724fa1461071d9d', 'request_id': 'req-18fb336c-d2e4-477a-97c4-5e335be50ba4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.247631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a9cc4f6e-7542-4567-9fb5-b57835cd6f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b647ab0681764239a5f003ad054de1b5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.255695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4126e23dd09a4fe2a724fa1461071d9d', 'request_id': 'req-18fb336c-d2e4-477a-97c4-5e335be50ba4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.274927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 865/1729] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:32 2022] DELETE /identity/v3/users/b647ab0681764239a5f003ad054de1b5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:32.277603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] The token cache is being invalidate because project 4126e23dd09a4fe2a724fa1461071d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:32.278710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.282417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570f57-7988-4819-bc03-39038162c435 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.282919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570f57-7988-4819-bc03-39038162c435 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.283749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570f57-7988-4819-bc03-39038162c435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.283909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-18fb336c-d2e4-477a-97c4-5e335be50ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:32.286345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 865/1730] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:31 2022] DELETE /identity/v3/projects/4126e23dd09a4fe2a724fa1461071d9d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:32.293803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a7330-43aa-451c-be8d-c1da89c0bb41 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.294425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a7330-43aa-451c-be8d-c1da89c0bb41 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.294425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a7330-43aa-451c-be8d-c1da89c0bb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.309415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-87570f57-7988-4819-bc03-39038162c435 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:32.310745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-87570f57-7988-4819-bc03-39038162c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:32.343638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-518a7330-43aa-451c-be8d-c1da89c0bb41 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:32.344920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-518a7330-43aa-451c-be8d-c1da89c0bb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:32.380521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 866/1731] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:32.391867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570f57-7988-4819-bc03-39038162c435 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:32.394926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-87570f57-7988-4819-bc03-39038162c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:32.440019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 866/1732] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:32.446850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.447468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.447958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.466552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570f57-7988-4819-bc03-39038162c435 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.476864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:32.477468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.478065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:32.478590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.479045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.479490 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.491834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 867/1733] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:32.494272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.496238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.496526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:32.507374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752a168-a925-4a52-8fd7-c3877847f491 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.516624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 867/1734] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:32.518368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.519111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.519389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.519660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.520383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:32.520753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.523081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.540764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 868/1735] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:32.542083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:32.573465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a912d96-adff-4ced-bc8e-3169af33e064 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.573465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.573465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.573465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.574015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.574015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.574158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.607626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 868/1736] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:32 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:32.611936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:32.624616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e067-6363-4342-87fc-f8805efafe67 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.910669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.911514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.911877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.912840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.914384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.917073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.941682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 869/1737] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:32 2022] POST /identity/v3/users => generated 283 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:32.946138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:32.958172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549ca2-cb62-4644-9254-abc8ea04a8bc admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:32.972798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:32.973529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:32.973977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:32.974517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:32.975545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:32.977919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:32.992477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 869/1738] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:32.996471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.015280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b0913-9fba-4f61-9beb-5ae1cb5dc4d7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.026807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.027491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.027772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.028766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.030884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.035643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.058379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 870/1739] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:33.063386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.075561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465a225-0f77-4d9b-a058-ac7ae7af12fd admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.079176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.081762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.088284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cef0b-1cef-4580-aeb4-038f06b729f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.089579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cef0b-1cef-4580-aeb4-038f06b729f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.092989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cef0b-1cef-4580-aeb4-038f06b729f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.093351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cef0b-1cef-4580-aeb4-038f06b729f0 admin admin] PATH_INFO: `/v3/projects/f58afcf4aabc4cc78e41863737552b8d/users/6205f595af854053a6217c3bc0811bcd/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.099095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83ac0e-8eac-4bdc-8a90-a89e5cf2076d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.108521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.109902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.111733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.112136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] PATH_INFO: `/v3/projects/788aaad0eeda440fa14a3acfa8ec52f2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.114385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=788aaad0eeda440fa14a3acfa8ec52f2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.118134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.120224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cef0b-1cef-4580-aeb4-038f06b729f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58afcf4aabc4cc78e41863737552b8d, user_id=6205f595af854053a6217c3bc0811bcd, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.125462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cef0b-1cef-4580-aeb4-038f06b729f0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.144011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 870/1740] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:33 2022] PUT /identity/v3/projects/f58afcf4aabc4cc78e41863737552b8d/users/6205f595af854053a6217c3bc0811bcd/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:33.155765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.165562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.166031 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.188234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788aaad0eeda440fa14a3acfa8ec52f2', 'request_id': 'req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.195704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788aaad0eeda440fa14a3acfa8ec52f2', 'request_id': 'req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.196670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] MFA Rules not processed for user `6205f595af854053a6217c3bc0811bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:33.198502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.203481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788aaad0eeda440fa14a3acfa8ec52f2', 'request_id': 'req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.223058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] The token cache is being invalidate because project 788aaad0eeda440fa14a3acfa8ec52f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:33.223932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.224732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.226700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 871/1741] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:33 2022] DELETE /identity/v3/projects/788aaad0eeda440fa14a3acfa8ec52f2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:33.231204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.233106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.293675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 871/1742] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:33.300324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.304879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.313784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3fa67-ddbc-4725-93f8-fce6d114d5ae admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.318737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b2bd-2d53-410a-b261-af6b88c47b55 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.319719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.320770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.321050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.321322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] PATH_INFO: `/v3/users/7cc470dcca1742c2ae5b366a709a218e` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.323155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc470dcca1742c2ae5b366a709a218e)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.323940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.324595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.324913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.325184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.325278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.325875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.327880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.358013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 872/1743] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:33 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:33.361998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.374976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc470dcca1742c2ae5b366a709a218e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.383178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc470dcca1742c2ae5b366a709a218e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.383914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc470dcca1742c2ae5b366a709a218e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.392758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc470dcca1742c2ae5b366a709a218e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.393610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc470dcca1742c2ae5b366a709a218e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.416325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 872/1744] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:33 2022] DELETE /identity/v3/users/7cc470dcca1742c2ae5b366a709a218e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:33.433912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81971ba8-65a4-4a3e-8c4d-8a9d42628f7e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.496382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.497108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.497378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.497662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.498354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.500267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.523326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 873/1745] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:33 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:33.525158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.526538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.540215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326804db-b7d3-452b-8faa-7b9115cc72ef admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.544802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.545530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.545899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.546243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.547044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.548750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.560847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 873/1746] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:33.568282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.580586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621bc36-b875-40af-bd5e-2259dfab6d5e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.586158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.587211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.587645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.587930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] PATH_INFO: `/v3/projects/98e2ad4a72fe4c82a137982066ee7785/users/2aeddc3e050840bbb64abb3dc9d66f79/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.604524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e2ad4a72fe4c82a137982066ee7785, user_id=2aeddc3e050840bbb64abb3dc9d66f79, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.607276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.622259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 874/1747] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:33 2022] PUT /identity/v3/projects/98e2ad4a72fe4c82a137982066ee7785/users/2aeddc3e050840bbb64abb3dc9d66f79/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:33.626214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.635915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149dca-f602-4bda-a268-aa7c9bb91f62 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.692217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.692910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.693192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.693466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.694236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.695812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.705544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 874/1748] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:33.709888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.721307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.722838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925ec78-6ca9-4b0d-820f-07068028925c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.723035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.728968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2c564-adb7-4429-b88d-432a59a45bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.730060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2c564-adb7-4429-b88d-432a59a45bca admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.730349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2c564-adb7-4429-b88d-432a59a45bca admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.730628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2c564-adb7-4429-b88d-432a59a45bca admin admin] PATH_INFO: `/v3/projects/98e2ad4a72fe4c82a137982066ee7785/users/2aeddc3e050840bbb64abb3dc9d66f79/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.732748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2c564-adb7-4429-b88d-432a59a45bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e2ad4a72fe4c82a137982066ee7785, user_id=2aeddc3e050840bbb64abb3dc9d66f79, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.736352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ff908-385d-4603-8513-574750f57225 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.742050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.742908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.743187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.743457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] PATH_INFO: `/v3/projects/4801f8314157432ca4f4d95c86d66177` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.744615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4801f8314157432ca4f4d95c86d66177)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.745371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2c564-adb7-4429-b88d-432a59a45bca admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.746558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.758204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 875/1749] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:33 2022] PUT /identity/v3/projects/98e2ad4a72fe4c82a137982066ee7785/users/2aeddc3e050840bbb64abb3dc9d66f79/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:33.768885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b775fab-957c-4b39-b545-961706f10301 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.769171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b775fab-957c-4b39-b545-961706f10301 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.769441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b775fab-957c-4b39-b545-961706f10301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.794831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4801f8314157432ca4f4d95c86d66177', 'request_id': 'req-983533f7-ba8b-40b5-9904-dd615465ab50'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.797873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3b775fab-957c-4b39-b545-961706f10301 None None] MFA Rules not processed for user `2aeddc3e050840bbb64abb3dc9d66f79`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:33.799119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3b775fab-957c-4b39-b545-961706f10301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.801739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4801f8314157432ca4f4d95c86d66177', 'request_id': 'req-983533f7-ba8b-40b5-9904-dd615465ab50'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.808123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4801f8314157432ca4f4d95c86d66177', 'request_id': 'req-983533f7-ba8b-40b5-9904-dd615465ab50'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.832889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] The token cache is being invalidate because project 4801f8314157432ca4f4d95c86d66177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:33.833285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.834007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-983533f7-ba8b-40b5-9904-dd615465ab50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:33.836695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 875/1750] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:33 2022] DELETE /identity/v3/projects/4801f8314157432ca4f4d95c86d66177 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:33.841199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715005-a19d-4adc-8dbd-f7588e0770c9 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.841546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715005-a19d-4adc-8dbd-f7588e0770c9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.841885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715005-a19d-4adc-8dbd-f7588e0770c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.860533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-82715005-a19d-4adc-8dbd-f7588e0770c9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:33.861850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-82715005-a19d-4adc-8dbd-f7588e0770c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.876685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 876/1751] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:33.889424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b775fab-957c-4b39-b545-961706f10301 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.890804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3b775fab-957c-4b39-b545-961706f10301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.921915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 876/1752] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:33.934945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.935238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.935515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.949093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b775fab-957c-4b39-b545-961706f10301 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.952662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:33.952772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.953306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.953583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.953851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.953948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:33.963110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 877/1753] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:33.963522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.964989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:33.966759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:33.975803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 877/1754] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:33.976249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca30648-9c3a-475e-b994-1de3d576cacf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:33.981049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:33.981846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:33.982154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:33.982456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:33.983221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:33.983469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:33.985071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.002403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 878/1755] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:51:34.008602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.019595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c08f4-cdf7-4829-b63e-01c201ae4eb9 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.026025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.026753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.027100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.027453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.028250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.030672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.061465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 878/1756] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:34 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:34.065736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.076280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afeac2-f1a1-48e0-b17a-3fbacf1071b7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.084408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.144873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.145644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.145989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.146597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.147785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.149738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.165946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdb7e9-cf51-4191-9014-a1e36600fabe admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.170262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.171094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.171341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.171622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.173681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:34.181706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 879/1757] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:34 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:34.185977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.201694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e23a1-4e9a-4ae5-8288-d368c31726a7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.206774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.207515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.207793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.208062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.208815 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.210972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.221489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 880/1758] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:34.225503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.235762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47275700-b250-4ccd-837a-61f1ff94f6c1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.240884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.241747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.242184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.242729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.243460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.245076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.254590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.255990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 881/1759] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:34.260039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072732ce-ebd0-435d-8ebc-f91ba4d72d0e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.261438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.270861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae50375-0e4a-49e8-8bdd-5cb5f1a34f89 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.271899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 879/1760] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:34.276328 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.277306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.277715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.278039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] PATH_INFO: `/v3/projects/e8a19690495e4569afad06ccfbe9cf79/users/7976d6844ffd494a9d1729c7f05de457/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.294043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a19690495e4569afad06ccfbe9cf79, user_id=7976d6844ffd494a9d1729c7f05de457, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.296693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.316209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 882/1761] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:34 2022] PUT /identity/v3/projects/e8a19690495e4569afad06ccfbe9cf79/users/7976d6844ffd494a9d1729c7f05de457/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:34.327291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.327728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.328216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.357391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] MFA Rules not processed for user `7976d6844ffd494a9d1729c7f05de457`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:34.358642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:34.431794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 880/1762] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:34.436159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.494712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6982961-eea3-4600-b436-cf247a0f207a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.499176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.499822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.500098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.500367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.501088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.502962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.527979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 883/1763] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:34 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:34.531945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.542731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad674e9-f0a3-4f6e-9436-989849c7d5eb None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.596341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.597063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.597344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.597614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.598295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.600191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.619696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 881/1764] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:34 2022] POST /identity/v3/users => generated 300 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:34.623626 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.634148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa20ee4-13d3-4326-a7fc-942bf556d3a4 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.638662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.639302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.639578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.639850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.640556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.642170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.652118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 884/1765] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:34.656259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.666360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e88ff9-1e82-4d4d-8b0f-47476f72ace0 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.670907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.671777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.672054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.672324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] PATH_INFO: `/v3/projects/90736cc413b9427ab12c2ba8e8ecb5a6/users/24ef6dae6f3d44c4a637f0d411e0f7ff/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.687148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90736cc413b9427ab12c2ba8e8ecb5a6, user_id=24ef6dae6f3d44c4a637f0d411e0f7ff, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.689837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.703917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 882/1766] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:34 2022] PUT /identity/v3/projects/90736cc413b9427ab12c2ba8e8ecb5a6/users/24ef6dae6f3d44c4a637f0d411e0f7ff/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:34.707741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.717912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ffdd-303b-4172-8b95-78ccd3279c77 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.772359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d5be-47ae-4f82-983e-c2a80840fd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.773118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d5be-47ae-4f82-983e-c2a80840fd1e admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.773401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d5be-47ae-4f82-983e-c2a80840fd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.773676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d5be-47ae-4f82-983e-c2a80840fd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.774394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245d5be-47ae-4f82-983e-c2a80840fd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.776007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245d5be-47ae-4f82-983e-c2a80840fd1e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.785990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 885/1767] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:34.790092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.800013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc13eb-146c-4681-8564-533bb0450aa4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.805145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.806005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.806279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.806547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] PATH_INFO: `/v3/projects/90736cc413b9427ab12c2ba8e8ecb5a6/users/24ef6dae6f3d44c4a637f0d411e0f7ff/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.808480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=90736cc413b9427ab12c2ba8e8ecb5a6, user_id=24ef6dae6f3d44c4a637f0d411e0f7ff, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:34.811169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:34.821482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 883/1768] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:34 2022] PUT /identity/v3/projects/90736cc413b9427ab12c2ba8e8ecb5a6/users/24ef6dae6f3d44c4a637f0d411e0f7ff/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:34.832806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.833221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.833511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:34.859481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] MFA Rules not processed for user `24ef6dae6f3d44c4a637f0d411e0f7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:34.860674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:34.924591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 886/1769] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:34.936000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:34.937720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:34.993254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4e687-77bc-4254-839a-fce75fa560bb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:34.996913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd3a91-6edc-48be-a8f0-6271dd8f246c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:34.997555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd3a91-6edc-48be-a8f0-6271dd8f246c service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:34.997835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd3a91-6edc-48be-a8f0-6271dd8f246c service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:34.998108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd3a91-6edc-48be-a8f0-6271dd8f246c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:35.008007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd3a91-6edc-48be-a8f0-6271dd8f246c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:35.009516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd3a91-6edc-48be-a8f0-6271dd8f246c service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:35.019961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 884/1770] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:35.094741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:35.157995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca67a-6109-4c8c-b001-dfed3e340513 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:35.163479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:35.164351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:35.164654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:35.164963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] PATH_INFO: `/v3/users/7976d6844ffd494a9d1729c7f05de457` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:35.166153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7976d6844ffd494a9d1729c7f05de457)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:35.168140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:35.211290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7976d6844ffd494a9d1729c7f05de457'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.217924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7976d6844ffd494a9d1729c7f05de457'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.218656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7976d6844ffd494a9d1729c7f05de457'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.226593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7976d6844ffd494a9d1729c7f05de457'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.227203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7976d6844ffd494a9d1729c7f05de457'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.244436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 887/1771] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:35 2022] DELETE /identity/v3/users/7976d6844ffd494a9d1729c7f05de457 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:35.248924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:35.249366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:35.249710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:35.266969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:35.268080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:35.332764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 885/1772] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:35.344074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:35.361610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:35.437964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c117bbf-c988-48cb-9c0a-53976a1ffba8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:35.444915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:35.446223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:35.447098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:35.447526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:35.461943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:35.463649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:35.474748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 888/1773] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:35.820261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:35.822060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:35.841693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7700a8d-0d51-4ae5-92fb-133790b27f59 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:35.846793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:35.847745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:35.848159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:35.848445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] PATH_INFO: `/v3/projects/e8a19690495e4569afad06ccfbe9cf79` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:35.849794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a19690495e4569afad06ccfbe9cf79)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:35.851842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:35.923067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a19690495e4569afad06ccfbe9cf79', 'request_id': 'req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.932551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a19690495e4569afad06ccfbe9cf79', 'request_id': 'req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.941208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a19690495e4569afad06ccfbe9cf79', 'request_id': 'req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.953015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] The token cache is being invalidate because project e8a19690495e4569afad06ccfbe9cf79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:35.953298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.953930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:35.960147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 886/1774] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:35 2022] DELETE /identity/v3/projects/e8a19690495e4569afad06ccfbe9cf79 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:35.966717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:35.982493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbc799-7cce-4c34-9450-58d8734a16fe service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.055634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.061079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.061409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.061652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] PATH_INFO: `/v3/users/24ef6dae6f3d44c4a637f0d411e0f7ff` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.062966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ef6dae6f3d44c4a637f0d411e0f7ff)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.065216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.114657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ef6dae6f3d44c4a637f0d411e0f7ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.123077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ef6dae6f3d44c4a637f0d411e0f7ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.124366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ef6dae6f3d44c4a637f0d411e0f7ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.140015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ef6dae6f3d44c4a637f0d411e0f7ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.140845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ea9449e3-a9c6-4c76-b57e-1cd87caa4ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ef6dae6f3d44c4a637f0d411e0f7ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.157525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 889/1775] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:35 2022] DELETE /identity/v3/users/24ef6dae6f3d44c4a637f0d411e0f7ff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:36.427977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.430161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:36.443613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf60a80-5ba3-48b3-979c-9e2ea0078e83 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.509237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.510130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.510408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.510681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] PATH_INFO: `/v3/projects/90736cc413b9427ab12c2ba8e8ecb5a6` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.513080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90736cc413b9427ab12c2ba8e8ecb5a6)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.515139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.575180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90736cc413b9427ab12c2ba8e8ecb5a6', 'request_id': 'req-767841c3-86d9-4082-8dca-2a4558461322'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.580818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90736cc413b9427ab12c2ba8e8ecb5a6', 'request_id': 'req-767841c3-86d9-4082-8dca-2a4558461322'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.590884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90736cc413b9427ab12c2ba8e8ecb5a6', 'request_id': 'req-767841c3-86d9-4082-8dca-2a4558461322'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.603535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] The token cache is being invalidate because project 90736cc413b9427ab12c2ba8e8ecb5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:36.603959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.604619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-767841c3-86d9-4082-8dca-2a4558461322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:36.609958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 887/1776] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:36 2022] DELETE /identity/v3/projects/90736cc413b9427ab12c2ba8e8ecb5a6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:36.615262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.616373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.616769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.634822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:36.636153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:36.694077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 890/1777] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:36.701908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.702200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.702485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.720590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:36.721821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:36.730306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 888/1778] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:36.734464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.744526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31640d-0b99-4e61-ab95-59b399a238fe None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.749053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.749695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.749961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.750225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.750929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:36.751225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.752816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.764009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 891/1779] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:36.768164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.777904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cfc4e-eca0-4bd0-9fe0-56d5676a3658 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.782428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.783081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.783375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.783644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.784608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.786503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.811523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 889/1780] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:36 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:36.815600 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.825326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a585e-402f-4e94-a6be-ea42c5eb4a1a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.879184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.879850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.880124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.880390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.881099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.883010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.902822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 892/1781] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:36 2022] POST /identity/v3/users => generated 291 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:36.906933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.922730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2454df7-66a2-4a20-b69c-4019fb24b58c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.927801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.928677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.929030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.929402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.930195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.932020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.942623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 890/1782] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:36.946721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.956421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377dfd4-6049-4723-ae6d-1be65259693e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.961672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47935c-67db-4228-9c1d-e1ec41f6cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.962335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47935c-67db-4228-9c1d-e1ec41f6cb99 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.962613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47935c-67db-4228-9c1d-e1ec41f6cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.962886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47935c-67db-4228-9c1d-e1ec41f6cb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:36.963759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47935c-67db-4228-9c1d-e1ec41f6cb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:36.965391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47935c-67db-4228-9c1d-e1ec41f6cb99 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:36.976243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 893/1783] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:36.980240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:36.990164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9478344-9c38-4a6d-bd4d-6fd382009c91 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:36.994742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:36.995607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:36.995907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:36.996154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] PATH_INFO: `/v3/projects/2fe3b1e69a824a66acdb158beb9f5972/users/8db738a8c3df4e749ccb9339995134a7/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.010053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe3b1e69a824a66acdb158beb9f5972, user_id=8db738a8c3df4e749ccb9339995134a7, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.012700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.026477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 891/1784] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:36 2022] PUT /identity/v3/projects/2fe3b1e69a824a66acdb158beb9f5972/users/8db738a8c3df4e749ccb9339995134a7/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:37.038170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.038475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.038822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.064574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] MFA Rules not processed for user `8db738a8c3df4e749ccb9339995134a7`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:37.065802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.141144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 894/1785] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:37.145147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.201860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.203330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.215686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6076f9e-fb10-4b39-9eed-d9421c61b381 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.221200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.221864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.222147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.222422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.223148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.225190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.254365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 892/1786] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:37 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:37.255361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.270072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953d1d-9362-4b52-8d8f-941a33a20f38 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.314642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca82dd-d2bd-4583-aaef-392875fe60e8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.319711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.320404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.320753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.321120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.321983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.323896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.324020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.324811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.325095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.325369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] PATH_INFO: `/v3/users/6205f595af854053a6217c3bc0811bcd` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.326571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] RBAC: Authorizing `identity:delete_user(user_id=6205f595af854053a6217c3bc0811bcd)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.328533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.343738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 893/1787] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:37 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:37.347645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.357279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbb805-2034-400b-94cb-4e9e73e0639a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.363044 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.363774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.364048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.364339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.365067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.366662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.375566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6205f595af854053a6217c3bc0811bcd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:37.376335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 894/1788] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:37.380609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.382720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6205f595af854053a6217c3bc0811bcd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:37.383494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6205f595af854053a6217c3bc0811bcd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:37.391394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6205f595af854053a6217c3bc0811bcd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:37.392476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-83a9bb9c-52e1-4330-8c38-094b9563eece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6205f595af854053a6217c3bc0811bcd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:37.395007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c481f-b7dc-4aa2-a6c9-8ed43f5d189d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.400282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.401228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.401504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.401776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] PATH_INFO: `/v3/projects/962053eb7e554a16841afeba8bee1fd1/users/e044a06fab2d46a6b944cdef7cb513e1/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.411027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 895/1789] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:37 2022] DELETE /identity/v3/users/6205f595af854053a6217c3bc0811bcd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:37.415361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.415647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.415921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.418435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=962053eb7e554a16841afeba8bee1fd1, user_id=e044a06fab2d46a6b944cdef7cb513e1, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.421286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.432804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:37.433980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.434748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 895/1790] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:37 2022] PUT /identity/v3/projects/962053eb7e554a16841afeba8bee1fd1/users/e044a06fab2d46a6b944cdef7cb513e1/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:37.438493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.440051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.489954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 896/1791] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:37.499317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.500619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.501722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25119565-3fce-4fb9-86f6-8d96c17f6821 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.506768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.507439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.507711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.507982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.508723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.510293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.519835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 896/1792] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:37.524752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.533967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfe0c-27ac-419a-a06f-ab9f4a78725b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.538910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80582b-fbc3-4c7f-91f9-e30e4da8da7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.539784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80582b-fbc3-4c7f-91f9-e30e4da8da7b admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.540068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80582b-fbc3-4c7f-91f9-e30e4da8da7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.540378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80582b-fbc3-4c7f-91f9-e30e4da8da7b admin admin] PATH_INFO: `/v3/projects/962053eb7e554a16841afeba8bee1fd1/users/e044a06fab2d46a6b944cdef7cb513e1/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.542424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80582b-fbc3-4c7f-91f9-e30e4da8da7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=962053eb7e554a16841afeba8bee1fd1, user_id=e044a06fab2d46a6b944cdef7cb513e1, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.545066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80582b-fbc3-4c7f-91f9-e30e4da8da7b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.557520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 897/1793] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:37 2022] PUT /identity/v3/projects/962053eb7e554a16841afeba8bee1fd1/users/e044a06fab2d46a6b944cdef7cb513e1/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:37.566800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7d467-4301-4ae2-89fb-6b8137f6d4d3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.569742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.570106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.570459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.599728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] MFA Rules not processed for user `e044a06fab2d46a6b944cdef7cb513e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:37.601308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.623507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.624727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.625214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.625726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.690239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 898/1794] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:37.703218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:37.704106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.704671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:37.705585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.716358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 897/1795] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:37.781100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ec83b-17c0-4df7-9cc8-05b4be3328ac None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:37.784999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:37.785639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:37.785924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:37.786199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:37.795892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:37.797465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:37.807610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 899/1796] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:38.075954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.077506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.091067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e46b1-a9a8-4c1a-8730-2f6e54154043 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.106549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.107537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.107805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.108153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] PATH_INFO: `/v3/projects/f58afcf4aabc4cc78e41863737552b8d` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.109428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58afcf4aabc4cc78e41863737552b8d)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.111420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.159609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58afcf4aabc4cc78e41863737552b8d', 'request_id': 'req-f645ef75-a70b-4316-8d2e-7331c0299342'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.167902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58afcf4aabc4cc78e41863737552b8d', 'request_id': 'req-f645ef75-a70b-4316-8d2e-7331c0299342'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.174141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58afcf4aabc4cc78e41863737552b8d', 'request_id': 'req-f645ef75-a70b-4316-8d2e-7331c0299342'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.188310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] The token cache is being invalidate because project f58afcf4aabc4cc78e41863737552b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:38.188620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.189291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.189967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.190975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 898/1797] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:38 2022] DELETE /identity/v3/projects/f58afcf4aabc4cc78e41863737552b8d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:38.194867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.196158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.199179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21e86f-b411-43e6-b594-622b78bbbabf service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.257607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e044a06fab2d46a6b944cdef7cb513e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '962053eb7e554a16841afeba8bee1fd1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.258367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.258718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.258901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645ef75-a70b-4316-8d2e-7331c0299342 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.259178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.259913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:38.260236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.261862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.264475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.265012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.265293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.265568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] PATH_INFO: `/v3/users/2aeddc3e050840bbb64abb3dc9d66f79` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.266770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aeddc3e050840bbb64abb3dc9d66f79)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.268705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.275574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 900/1798] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:51:38.279409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.281104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.295830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14bfde-2d84-4566-af5c-1517fa7a605e tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.301046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e044a06fab2d46a6b944cdef7cb513e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '962053eb7e554a16841afeba8bee1fd1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.301765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.302175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.302405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.303076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.305310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.321198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aeddc3e050840bbb64abb3dc9d66f79'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.330398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aeddc3e050840bbb64abb3dc9d66f79'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.331138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aeddc3e050840bbb64abb3dc9d66f79'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.337170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 901/1799] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:38 2022] POST /identity/v3/projects => generated 352 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:38.342069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aeddc3e050840bbb64abb3dc9d66f79'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.342735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aeddc3e050840bbb64abb3dc9d66f79'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.349222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.351354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.360461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 899/1800] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:38 2022] DELETE /identity/v3/users/2aeddc3e050840bbb64abb3dc9d66f79 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:38.414758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55af0-fdef-4f21-8bf2-4f6eaea50665 tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.471505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e044a06fab2d46a6b944cdef7cb513e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '962053eb7e554a16841afeba8bee1fd1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.472177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.472456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.472843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.473563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:38.473869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.476188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.487376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 902/1801] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:38.529552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.541123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd585134-6748-49e0-ad3a-ecb8f0a8330a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.545865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e044a06fab2d46a6b944cdef7cb513e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '962053eb7e554a16841afeba8bee1fd1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.546915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.547267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.547549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] PATH_INFO: `/v3/projects/2ce592ebeab846d58a29049075df28b2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.548754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2ce592ebeab846d58a29049075df28b2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.550667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.597070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce592ebeab846d58a29049075df28b2', 'request_id': 'req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.603108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce592ebeab846d58a29049075df28b2', 'request_id': 'req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.609322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce592ebeab846d58a29049075df28b2', 'request_id': 'req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.623120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] The token cache is being invalidate because project 2ce592ebeab846d58a29049075df28b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:38.623344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.623994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9f0edf9d-7a23-4da9-81f5-894fa209a2ba tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.625588 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 900/1802] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:38 2022] DELETE /identity/v3/projects/2ce592ebeab846d58a29049075df28b2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:38.642588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.644347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.712294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1715-c90e-409f-ba72-675ea69b657a tempest-VolumeQuotaClassesTest-1293237036 tempest-VolumeQuotaClassesTest-1293237036-project-admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.783625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.784902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.785185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.785554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] PATH_INFO: `/v3/projects/98e2ad4a72fe4c82a137982066ee7785` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.787372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e2ad4a72fe4c82a137982066ee7785)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.789506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:38.852555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e2ad4a72fe4c82a137982066ee7785', 'request_id': 'req-8b5df116-255d-4db3-bfc6-431f86e412e1'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.857055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e2ad4a72fe4c82a137982066ee7785', 'request_id': 'req-8b5df116-255d-4db3-bfc6-431f86e412e1'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.864869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e2ad4a72fe4c82a137982066ee7785', 'request_id': 'req-8b5df116-255d-4db3-bfc6-431f86e412e1'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.878004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] The token cache is being invalidate because project 98e2ad4a72fe4c82a137982066ee7785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:38.878237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.878866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:38.880434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 903/1803] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:38 2022] DELETE /identity/v3/projects/98e2ad4a72fe4c82a137982066ee7785 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:38.885982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378687-93da-4cc6-90c8-b4bbfe779c87 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.886309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378687-93da-4cc6-90c8-b4bbfe779c87 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.886614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378687-93da-4cc6-90c8-b4bbfe779c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.909598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-45378687-93da-4cc6-90c8-b4bbfe779c87 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:38.909598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-45378687-93da-4cc6-90c8-b4bbfe779c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.920134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:38.921538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:38.974299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 901/1804] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:38.982017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.982313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.982592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.988747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5df116-255d-4db3-bfc6-431f86e412e1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:38.994328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:38.995197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:38.995521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:38.995795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] PATH_INFO: `/v3/users/8db738a8c3df4e749ccb9339995134a7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:38.997029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db738a8c3df4e749ccb9339995134a7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:38.999593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.001665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:39.001905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:39.013752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 902/1805] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:39.015248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.026244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21053a5-6608-491f-a02c-d82da6ea456e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.031456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.031675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.031949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.032224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.033053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:39.033364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.035211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.047615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 903/1806] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:39.051908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.059793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db738a8c3df4e749ccb9339995134a7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.060515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db738a8c3df4e749ccb9339995134a7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.063041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b062-1091-453d-b9a6-1ba85166456e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.067960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.068116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db738a8c3df4e749ccb9339995134a7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.069469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.070032 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.070461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.071287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.073394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.078262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db738a8c3df4e749ccb9339995134a7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.078893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f793f28e-e411-4291-a914-d1ceeb9f71ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db738a8c3df4e749ccb9339995134a7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.097601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 904/1807] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:38 2022] DELETE /identity/v3/users/8db738a8c3df4e749ccb9339995134a7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:39.101586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 904/1808] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:39.105470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.105771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.105938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.106068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.107890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:39.126373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:39.127786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:39.176213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c2213-0714-4506-8406-28ba2e4063e6 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.181365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.182203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.182499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.182778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.183525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.185507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.203064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 905/1809] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:39.207714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 905/1810] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:39.211410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.212953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.214738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:39.221438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80274e-7879-40c4-8285-1f6abf555ce5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.228119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.228900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.229253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.229600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.230377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.231999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.243230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 906/1811] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:39.247409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.272526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3909d-b100-4d2b-8085-68673ec764e5 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.286496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.287176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.287553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.287757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.288486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.290138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.328214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53100517-8782-4d5c-9aff-0057b5b32375 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.328641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 907/1812] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:39.329611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.333040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbc1fd-2d40-4b9c-8b9e-64a6da3dd431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.333957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbc1fd-2d40-4b9c-8b9e-64a6da3dd431 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.334247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbc1fd-2d40-4b9c-8b9e-64a6da3dd431 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.334766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbc1fd-2d40-4b9c-8b9e-64a6da3dd431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.351912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8953-f07c-4c32-af33-a0578130d188 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.359322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbc1fd-2d40-4b9c-8b9e-64a6da3dd431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.362883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbc1fd-2d40-4b9c-8b9e-64a6da3dd431 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.363812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.364820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.365107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.367188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] PATH_INFO: `/v3/projects/217a109092c94befabed4b8951c0cbf3/users/b7675f1fadde4570a2edc7dc41da56d4/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.387544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217a109092c94befabed4b8951c0cbf3, user_id=b7675f1fadde4570a2edc7dc41da56d4, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.388932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 906/1813] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:39.390449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.408837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 908/1814] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:39 2022] PUT /identity/v3/projects/217a109092c94befabed4b8951c0cbf3/users/b7675f1fadde4570a2edc7dc41da56d4/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:39.420610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.420932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.421826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.468985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] MFA Rules not processed for user `b7675f1fadde4570a2edc7dc41da56d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:39.470375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:39.546894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 907/1815] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:39.549167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.629217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a613-d3ab-4499-b4ff-d5b8558055a1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.633874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.634532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.634840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.635112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.635798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.637729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.684822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 909/1816] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:39.685883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.719940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e0299-ed22-48d6-805c-59b5f2845031 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.784745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.785415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.785692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.785961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.786720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.789239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.810115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 908/1817] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:39.814088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.824434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b08a7b-ebb7-41e5-ae36-1a35b1e19706 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.829051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.829699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.830001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.830246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.830991 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.832605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.838890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.840845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:39.846718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 910/1818] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:39.849940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.856035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c5a19-c258-44d5-a832-214c83f53ac9 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.860836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.861398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db550c7f-b570-49f6-b2ac-3ef8b02f85fc admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.861704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.861985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.862258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] PATH_INFO: `/v3/projects/2fe3b1e69a824a66acdb158beb9f5972` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.863537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe3b1e69a824a66acdb158beb9f5972)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.865470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.866655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.867316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.867593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.867893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.868562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.870279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.884139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 911/1819] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:39.887015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:39.897801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb297-f888-4aac-ac62-aedaa5e77195 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:39.902870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128573ef-721c-40ae-b1ba-f90a8cb6fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:39.903839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128573ef-721c-40ae-b1ba-f90a8cb6fba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.904186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128573ef-721c-40ae-b1ba-f90a8cb6fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.904520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128573ef-721c-40ae-b1ba-f90a8cb6fba5 admin admin] PATH_INFO: `/v3/projects/06105aec77084347981d8d4bc25f6c58/users/2ea52a4a8d434a88956ba782cceb8639/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.920594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128573ef-721c-40ae-b1ba-f90a8cb6fba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06105aec77084347981d8d4bc25f6c58, user_id=2ea52a4a8d434a88956ba782cceb8639, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:39.923936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe3b1e69a824a66acdb158beb9f5972', 'request_id': 'req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.924926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128573ef-721c-40ae-b1ba-f90a8cb6fba5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:39.931040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe3b1e69a824a66acdb158beb9f5972', 'request_id': 'req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.939032 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe3b1e69a824a66acdb158beb9f5972', 'request_id': 'req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:39.942583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 912/1820] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:39 2022] PUT /identity/v3/projects/06105aec77084347981d8d4bc25f6c58/users/2ea52a4a8d434a88956ba782cceb8639/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:39.954569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:39.955787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:39.956212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:39.977501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] The token cache is being invalidate because project 2fe3b1e69a824a66acdb158beb9f5972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:40.003257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.005777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.014536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 909/1821] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:39 2022] DELETE /identity/v3/projects/2fe3b1e69a824a66acdb158beb9f5972 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:40.025000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.031742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.072209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] MFA Rules not processed for user `2ea52a4a8d434a88956ba782cceb8639`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:40.073632 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.118958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bfdb1-9da9-47f5-982b-d13ea06221c3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.226543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.227470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.227813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.228089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] PATH_INFO: `/v3/users/e044a06fab2d46a6b944cdef7cb513e1` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.230762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e044a06fab2d46a6b944cdef7cb513e1)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.237298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.245248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 913/1822] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:40.249800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.251180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.279454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1167d7-d449-444c-bd0c-9ceb6ab16e5b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.305453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.306173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.306464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.306743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.307452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.309412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.312137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e044a06fab2d46a6b944cdef7cb513e1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.326447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e044a06fab2d46a6b944cdef7cb513e1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.327380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e044a06fab2d46a6b944cdef7cb513e1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.357809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 914/1823] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:40 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:40.358770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.359572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e044a06fab2d46a6b944cdef7cb513e1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.360732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e044a06fab2d46a6b944cdef7cb513e1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:40.389190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 910/1824] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:40 2022] DELETE /identity/v3/users/e044a06fab2d46a6b944cdef7cb513e1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:40.450022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1099470-f414-42cd-aaf6-285de104457a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.520696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.521242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.521546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.521854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.523125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.525301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.552666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 915/1825] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:40 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:40.556748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.558165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.572719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9c750-322a-47e5-b0d7-52b5a647e53a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.578367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.578942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.579290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.579631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.580665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.582185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.594326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 911/1826] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:40.597743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.608160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cc8ae-0a69-435f-88c6-66a2bb797380 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.613216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.614118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.614396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.614664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] PATH_INFO: `/v3/projects/866ed583b4cb41a080036be99a9334b6/users/4cead914c57a45bb9bbedd7efc1d6b00/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.632205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] RBAC: Authorizing `identity:create_grant(project_id=866ed583b4cb41a080036be99a9334b6, user_id=4cead914c57a45bb9bbedd7efc1d6b00, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.635195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.650796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 916/1827] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:40 2022] PUT /identity/v3/projects/866ed583b4cb41a080036be99a9334b6/users/4cead914c57a45bb9bbedd7efc1d6b00/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:40.654604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.665848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a6bc4-1120-4b80-802e-f14085471a6d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.722791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfcd64-da73-4821-96e7-61fdbe8ae5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.723420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfcd64-da73-4821-96e7-61fdbe8ae5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.723632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfcd64-da73-4821-96e7-61fdbe8ae5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.723979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfcd64-da73-4821-96e7-61fdbe8ae5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.724612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfcd64-da73-4821-96e7-61fdbe8ae5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.726851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfcd64-da73-4821-96e7-61fdbe8ae5f6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.737751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 912/1828] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:40.742336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.757739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05badb51-3be9-4c79-9ef1-1b147323206a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.762271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.763292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.763610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.763958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] PATH_INFO: `/v3/projects/866ed583b4cb41a080036be99a9334b6/users/4cead914c57a45bb9bbedd7efc1d6b00/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.766347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=866ed583b4cb41a080036be99a9334b6, user_id=4cead914c57a45bb9bbedd7efc1d6b00, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.768822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.780502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 917/1829] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:40 2022] PUT /identity/v3/projects/866ed583b4cb41a080036be99a9334b6/users/4cead914c57a45bb9bbedd7efc1d6b00/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:40.790967 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.791760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.792170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.819743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] MFA Rules not processed for user `4cead914c57a45bb9bbedd7efc1d6b00`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:40.820943 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.891119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 913/1830] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:40.902913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.904699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.905659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:40.913704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:40.972349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac5394-bc83-47ce-adad-3c2b80d2c99d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.976681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604cc3b-53f9-4b75-b3fc-b2b3045a4ca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.977284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604cc3b-53f9-4b75-b3fc-b2b3045a4ca8 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.977569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604cc3b-53f9-4b75-b3fc-b2b3045a4ca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.977845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604cc3b-53f9-4b75-b3fc-b2b3045a4ca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.978160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa778804-b7da-4e07-bab9-a8c557ccb001 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:40.982730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:40.983648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:40.983997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:40.984333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] PATH_INFO: `/v3/projects/962053eb7e554a16841afeba8bee1fd1` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:40.985637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=962053eb7e554a16841afeba8bee1fd1)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.986767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604cc3b-53f9-4b75-b3fc-b2b3045a4ca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:40.987665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.988179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604cc3b-53f9-4b75-b3fc-b2b3045a4ca8 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:40.998240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 918/1831] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:41.039777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962053eb7e554a16841afeba8bee1fd1', 'request_id': 'req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:41.045465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962053eb7e554a16841afeba8bee1fd1', 'request_id': 'req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:41.053093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962053eb7e554a16841afeba8bee1fd1', 'request_id': 'req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:41.070979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] The token cache is being invalidate because project 962053eb7e554a16841afeba8bee1fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:41.071330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:41.072301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a4c6c303-d8e9-4c2f-b3dd-0eb7fa4164b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:41.078099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 914/1832] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:40 2022] DELETE /identity/v3/projects/962053eb7e554a16841afeba8bee1fd1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:41.080325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.080874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.081354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.114748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:41.116561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.209869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 919/1833] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:41.209979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.210061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.210333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.232122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:41.232122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.244681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 915/1834] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:41.247004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.258288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e7c84-58cf-4f01-9b22-01c4d9bf2a74 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.264032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.264856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.265136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.265407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.266161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:41.266426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.268255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.285286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 920/1835] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:41.290156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.299441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bdb9a-145f-4fab-aa87-5442d9cab023 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.304498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc700-3dd9-4afc-b344-d2239b488228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.305334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc700-3dd9-4afc-b344-d2239b488228 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.305618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc700-3dd9-4afc-b344-d2239b488228 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.305874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc700-3dd9-4afc-b344-d2239b488228 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.306574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fc700-3dd9-4afc-b344-d2239b488228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.308451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fc700-3dd9-4afc-b344-d2239b488228 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.343835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 916/1836] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:41 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:41.347993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.362542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7fd0a-ec83-4a16-a687-0b66368f99ea admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.464339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.465053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.465332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.465608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.466472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.468775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.490142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 921/1837] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:41 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:41.493965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f3f06-e7eb-4511-adbb-511cefdd66f5 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.493965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f3f06-e7eb-4511-adbb-511cefdd66f5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.494153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.494251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f3f06-e7eb-4511-adbb-511cefdd66f5 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.495957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 917/1838] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:51:41.511538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2880-b1c9-46fd-8dc4-316efd76c618 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.513208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.513832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.514400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.514510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.514734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.515800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.516343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.517405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.519355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.541883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 922/1839] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:41.547326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.560550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:41.561777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.569006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1d0cb-fec5-419e-b7c5-e1145efc8521 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.589722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.590544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.590844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.593595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.594351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.595963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.611627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 923/1840] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:41.612504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.623181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3c42-ae63-4e1b-9898-62b3c2aa2191 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.636818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893b136-16de-4159-a529-b8c965370169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.638315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893b136-16de-4159-a529-b8c965370169 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.638970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893b136-16de-4159-a529-b8c965370169 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.638970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893b136-16de-4159-a529-b8c965370169 admin admin] PATH_INFO: `/v3/projects/7a1f37f5754b44f5a2f2a85b45442782/users/766f124af4444076a76faa6049c2ddfd/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.664212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893b136-16de-4159-a529-b8c965370169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1f37f5754b44f5a2f2a85b45442782, user_id=766f124af4444076a76faa6049c2ddfd, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.667739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893b136-16de-4159-a529-b8c965370169 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.695191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 924/1841] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:41 2022] PUT /identity/v3/projects/7a1f37f5754b44f5a2f2a85b45442782/users/766f124af4444076a76faa6049c2ddfd/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:41.705438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.706531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.706962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.711050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 918/1842] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:41.722410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.723046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.747125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] MFA Rules not processed for user `766f124af4444076a76faa6049c2ddfd`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:41.748346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.801545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d8df9-9c4f-4387-95c8-7ead80afd1ff None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.827001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 925/1843] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:41.827936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.829454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.867612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.868423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.868882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.869200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.871102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:41.926994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500fb5c-9d76-4106-9331-4b2f608303ab None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:41.933499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:41.934162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:41.934441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:41.934785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:41.944334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.946103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.946726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:41.948145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:41.957336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 926/1844] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:41.957608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 919/1845] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:51:41.961713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:41.964262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:42.028194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606d37-cfd6-47e7-b4bb-715a81dd7cb1 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:42.032078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:42.032763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:42.033040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:42.033310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:42.042413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:42.043823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:42.054689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 927/1846] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:46.333453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:46.335512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:46.349066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6f06-9158-4e3e-a2be-ab14e8f09b13 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:46.353599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:46.354364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:46.354699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:46.355033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:46.356990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:46.435977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:46.437483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eae6a5-0ec6-491d-be84-ebbb686289a7 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:46.448467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 920/1847] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:49.279673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:49.281090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:49.343871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53042-bea9-4b91-9776-41cb87116807 service glance] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:49.348396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:49.349283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:49.350084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:49.350354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] PATH_INFO: `/v3/users/b7675f1fadde4570a2edc7dc41da56d4` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:49.351531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7675f1fadde4570a2edc7dc41da56d4)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:49.353462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:49.396596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7675f1fadde4570a2edc7dc41da56d4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.397264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7675f1fadde4570a2edc7dc41da56d4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.403773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7675f1fadde4570a2edc7dc41da56d4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.410975 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7675f1fadde4570a2edc7dc41da56d4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.411598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7675f1fadde4570a2edc7dc41da56d4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.429110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 928/1848] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:49 2022] DELETE /identity/v3/users/b7675f1fadde4570a2edc7dc41da56d4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:49.433568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:49.434022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:49.434132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:49.450876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:49.452059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:49.507627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 921/1849] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:49.516980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:49.518344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:49.573055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f45139-4655-4a1b-851b-a436784dec45 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:49.576653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:49.577297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:49.577579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:49.577859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:49.587137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:49.588543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:49.597700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 929/1850] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:49.855471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:49.856962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:49.868958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd01310-3612-4172-924a-9d50e14bbc10 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:49.874171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:49.875706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:49.876383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:49.876705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] PATH_INFO: `/v3/projects/217a109092c94befabed4b8951c0cbf3` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:49.877903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217a109092c94befabed4b8951c0cbf3)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:49.879890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:49.928590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217a109092c94befabed4b8951c0cbf3', 'request_id': 'req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.941691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217a109092c94befabed4b8951c0cbf3', 'request_id': 'req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.953689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217a109092c94befabed4b8951c0cbf3', 'request_id': 'req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.969299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] The token cache is being invalidate because project 217a109092c94befabed4b8951c0cbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:49.969644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.970293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:49.972222 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 922/1851] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:49 2022] DELETE /identity/v3/projects/217a109092c94befabed4b8951c0cbf3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:49.975985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:49.985650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836584b-a23b-47ff-b678-85a8363a1f7d service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:50.045413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:50.046301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:50.046584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:50.046860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] PATH_INFO: `/v3/users/2ea52a4a8d434a88956ba782cceb8639` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:50.048057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea52a4a8d434a88956ba782cceb8639)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:50.050005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:50.107204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea52a4a8d434a88956ba782cceb8639'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.107849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea52a4a8d434a88956ba782cceb8639'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.114864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea52a4a8d434a88956ba782cceb8639'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.122503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea52a4a8d434a88956ba782cceb8639'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.123312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea52a4a8d434a88956ba782cceb8639'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.143444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 930/1852] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:49 2022] DELETE /identity/v3/users/2ea52a4a8d434a88956ba782cceb8639 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:50.484654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:50.486125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:50.499397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372feb1-3404-4b13-8f4d-4f3a6bcbbfe8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:50.558146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:50.558998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:50.559273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:50.559540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] PATH_INFO: `/v3/projects/06105aec77084347981d8d4bc25f6c58` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:50.560731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=06105aec77084347981d8d4bc25f6c58)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:50.562608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:50.606623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06105aec77084347981d8d4bc25f6c58', 'request_id': 'req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.611965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06105aec77084347981d8d4bc25f6c58', 'request_id': 'req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.618425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06105aec77084347981d8d4bc25f6c58', 'request_id': 'req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.631634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] The token cache is being invalidate because project 06105aec77084347981d8d4bc25f6c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:50.631914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.632599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.634151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 923/1853] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:50 2022] DELETE /identity/v3/projects/06105aec77084347981d8d4bc25f6c58 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:50.638247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:50.639729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:50.652980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398867a0-3dc6-4675-bf13-989495726e29 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:50.710009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:50.710876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:50.711159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:50.711435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] PATH_INFO: `/v3/users/4cead914c57a45bb9bbedd7efc1d6b00` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:50.712611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cead914c57a45bb9bbedd7efc1d6b00)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:50.714584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:50.758576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cead914c57a45bb9bbedd7efc1d6b00'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.759223 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cead914c57a45bb9bbedd7efc1d6b00'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.765886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cead914c57a45bb9bbedd7efc1d6b00'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.773417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cead914c57a45bb9bbedd7efc1d6b00'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.774036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2705c7b1-eb12-4271-a762-428c3861892b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cead914c57a45bb9bbedd7efc1d6b00'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:50.791089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 931/1854] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:50 2022] DELETE /identity/v3/users/4cead914c57a45bb9bbedd7efc1d6b00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:51.077929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.079338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:51.094259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f627fe1-eccd-4862-aa3c-d62744ad0dcf admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.165858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.166993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.167285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.167559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] PATH_INFO: `/v3/projects/866ed583b4cb41a080036be99a9334b6` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.168932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=866ed583b4cb41a080036be99a9334b6)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.170984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.228597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '866ed583b4cb41a080036be99a9334b6', 'request_id': 'req-d2823781-574f-4134-bd27-a4a6da7226b9'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:51.234825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '866ed583b4cb41a080036be99a9334b6', 'request_id': 'req-d2823781-574f-4134-bd27-a4a6da7226b9'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:51.242220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '866ed583b4cb41a080036be99a9334b6', 'request_id': 'req-d2823781-574f-4134-bd27-a4a6da7226b9'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:51.257113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] The token cache is being invalidate because project 866ed583b4cb41a080036be99a9334b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:51.257398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:51.259902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d2823781-574f-4134-bd27-a4a6da7226b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:51.261529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 924/1855] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:51 2022] DELETE /identity/v3/projects/866ed583b4cb41a080036be99a9334b6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:51.267477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.267845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.268213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.293866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:51.295179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:51.357719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 932/1856] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:51.365842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.366290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.366729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.385921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:51.387088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:51.394824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 925/1857] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:51.399500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.408937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6043b-efd3-4288-b846-6c0b92a987c9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.413736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.414383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.414662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.414936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.415650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:51.415952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.417566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.428494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 933/1858] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:51.432342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.441068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56982-d6bb-43ef-a480-581cb35fb005 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.446067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.446693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.446970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.447240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.447913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.449809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.474016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 926/1859] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/projects => generated 385 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:51.478257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.488320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981885bd-96a4-41cb-85c4-1f6e1f2250e8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.546048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.546712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.547246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.547525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.548210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.550163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.570299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 934/1860] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:51.574114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.584973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfb85-a59d-4430-93c6-4dbef3fe076d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.589681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.590338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.590616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.590888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.591613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.593224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.603441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 927/1861] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:51.607443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.620897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a13c8c-3f9a-4260-ad1f-69fe90c55b26 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.629072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4a5de-9201-486d-b9a7-80dc45c9b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.629865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a5de-9201-486d-b9a7-80dc45c9b8cd admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.632384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a5de-9201-486d-b9a7-80dc45c9b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.632699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a5de-9201-486d-b9a7-80dc45c9b8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.633428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4a5de-9201-486d-b9a7-80dc45c9b8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.635024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4a5de-9201-486d-b9a7-80dc45c9b8cd admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.649716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 935/1862] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:51.653871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.664147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e01ab-b858-48ee-8e06-6a85f73eb175 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.668715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.669577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.669854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.670125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] PATH_INFO: `/v3/projects/b349c408951c4611ad949fab3c660f3f/users/cadced7fc8bd4ef68783f4bf5abfbb6f/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.684048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b349c408951c4611ad949fab3c660f3f, user_id=cadced7fc8bd4ef68783f4bf5abfbb6f, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.686709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.699483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 928/1863] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:51 2022] PUT /identity/v3/projects/b349c408951c4611ad949fab3c660f3f/users/cadced7fc8bd4ef68783f4bf5abfbb6f/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:51.710986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.711268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.711542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.737296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] MFA Rules not processed for user `cadced7fc8bd4ef68783f4bf5abfbb6f`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:51.738470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:51.799535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 936/1864] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:51.804466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.862358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3a624-9cde-46aa-8fb8-403e5eca293b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.867371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.868018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.868298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.868573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.869308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.871191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.895272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 929/1865] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:51.899095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:51.909647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e566270-b965-4a9b-9b75-9f6eff12e437 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:51.964184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:51.964872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:51.965150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:51.965419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:51.966197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:51.968093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:51.987908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 937/1866] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:51 2022] POST /identity/v3/users => generated 300 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:51.992299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:52.002476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce560-fbaf-46a6-bb44-b49b99c7c057 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:52.007759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:52.008484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:52.008866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:52.009213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:52.010011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:52.011684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:52.022365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 930/1867] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:52.026558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:52.036650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f395f0-1017-4655-8222-608a83f3c4f6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:52.041418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:52.042341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:52.042705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:52.043033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] PATH_INFO: `/v3/projects/a444a4e5ba2d4678bd076919e7996ccf/users/37a86a0a67e54ba6b64debbc2202f294/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:52.057809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a444a4e5ba2d4678bd076919e7996ccf, user_id=37a86a0a67e54ba6b64debbc2202f294, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:52.061070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:52.075412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 938/1868] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:52 2022] PUT /identity/v3/projects/a444a4e5ba2d4678bd076919e7996ccf/users/37a86a0a67e54ba6b64debbc2202f294/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:52.079379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:52.089739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec924cd-b264-4e9d-9f74-a54710b9daba admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:52.152304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc69aa-9acd-4926-8956-544e7f528c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:52.153182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc69aa-9acd-4926-8956-544e7f528c5c admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:52.153465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc69aa-9acd-4926-8956-544e7f528c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:52.153770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc69aa-9acd-4926-8956-544e7f528c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:52.154496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc69aa-9acd-4926-8956-544e7f528c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:52.156112 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc69aa-9acd-4926-8956-544e7f528c5c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:52.167104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 931/1869] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:52.171325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:52.182128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee78941-5f84-4c59-bf10-2eb953907f25 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:52.187232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:52.188188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:52.188510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:52.189068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] PATH_INFO: `/v3/projects/a444a4e5ba2d4678bd076919e7996ccf/users/37a86a0a67e54ba6b64debbc2202f294/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:52.191104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a444a4e5ba2d4678bd076919e7996ccf, user_id=37a86a0a67e54ba6b64debbc2202f294, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:52.193938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:52.204763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 939/1870] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:52 2022] PUT /identity/v3/projects/a444a4e5ba2d4678bd076919e7996ccf/users/37a86a0a67e54ba6b64debbc2202f294/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:52.216904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:52.217433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:52.218273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:52.248661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] MFA Rules not processed for user `37a86a0a67e54ba6b64debbc2202f294`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:52.249839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:52.336079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 932/1871] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:52.343753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:52.345547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:52.409747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264675-3206-4364-80ce-cb68f19ba25d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:52.414054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196afd7-ffdb-4956-aa80-595eddfc482a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:52.414700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196afd7-ffdb-4956-aa80-595eddfc482a service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:52.414980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196afd7-ffdb-4956-aa80-595eddfc482a service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:52.415255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196afd7-ffdb-4956-aa80-595eddfc482a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:52.425004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196afd7-ffdb-4956-aa80-595eddfc482a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:52.426465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196afd7-ffdb-4956-aa80-595eddfc482a service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:52.437196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 940/1872] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:52.984586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:53.044772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d118e-0912-4fc7-bea5-d1589599576a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:53.049249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:53.050101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:53.050379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:53.050645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] PATH_INFO: `/v3/users/cadced7fc8bd4ef68783f4bf5abfbb6f` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:53.051812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadced7fc8bd4ef68783f4bf5abfbb6f)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:53.053743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:53.098454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadced7fc8bd4ef68783f4bf5abfbb6f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.104881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadced7fc8bd4ef68783f4bf5abfbb6f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.105522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadced7fc8bd4ef68783f4bf5abfbb6f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.112053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadced7fc8bd4ef68783f4bf5abfbb6f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.112672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadced7fc8bd4ef68783f4bf5abfbb6f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.129418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 933/1873] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:52 2022] DELETE /identity/v3/users/cadced7fc8bd4ef68783f4bf5abfbb6f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:53.134417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:53.134892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:53.134984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:53.153123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:53.154432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:53.212010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 941/1874] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:53.221757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:53.223274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:53.292018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570bb1-fe1b-43cd-80e2-5f6202d9ed3f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:53.295912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:53.296678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:53.296830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:53.297106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:53.306775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:53.308190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:53.318040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 934/1875] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:53.628955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:53.631466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:53.643032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9475-8b6a-4e3d-a635-eb5056051763 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:53.647944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:53.648927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:53.649321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:53.649712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] PATH_INFO: `/v3/projects/b349c408951c4611ad949fab3c660f3f` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:53.661476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b349c408951c4611ad949fab3c660f3f)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:53.663990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:53.713757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b349c408951c4611ad949fab3c660f3f', 'request_id': 'req-80995623-3598-4704-8ebc-243ab3a38456'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.718811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b349c408951c4611ad949fab3c660f3f', 'request_id': 'req-80995623-3598-4704-8ebc-243ab3a38456'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.726708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b349c408951c4611ad949fab3c660f3f', 'request_id': 'req-80995623-3598-4704-8ebc-243ab3a38456'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.737607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] The token cache is being invalidate because project b349c408951c4611ad949fab3c660f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:53.737893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.738522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.740715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 942/1876] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:53 2022] DELETE /identity/v3/projects/b349c408951c4611ad949fab3c660f3f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:53.745526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:53.755092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2ea11-a561-4dcf-a20d-be8bc1834e75 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:53.813036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:53.813903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:53.814182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:53.814478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] PATH_INFO: `/v3/users/37a86a0a67e54ba6b64debbc2202f294` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:53.815639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a86a0a67e54ba6b64debbc2202f294)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:53.817654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:53.864582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a86a0a67e54ba6b64debbc2202f294'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.873254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a86a0a67e54ba6b64debbc2202f294'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.873254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a86a0a67e54ba6b64debbc2202f294'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.880950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a86a0a67e54ba6b64debbc2202f294'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.882078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2bb6337a-c0f3-4123-9415-8df28d4d3547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a86a0a67e54ba6b64debbc2202f294'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:53.902391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 935/1877] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:53 2022] DELETE /identity/v3/users/37a86a0a67e54ba6b64debbc2202f294 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:51:54.255730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.257343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:54.274246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80995623-3598-4704-8ebc-243ab3a38456 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.340618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.342821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.343106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.343389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] PATH_INFO: `/v3/projects/a444a4e5ba2d4678bd076919e7996ccf` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.344610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a444a4e5ba2d4678bd076919e7996ccf)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.346548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:54.401363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a444a4e5ba2d4678bd076919e7996ccf', 'request_id': 'req-43bcb251-be6a-4071-8f19-ca2d506d6af5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:54.407938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a444a4e5ba2d4678bd076919e7996ccf', 'request_id': 'req-43bcb251-be6a-4071-8f19-ca2d506d6af5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:54.420651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a444a4e5ba2d4678bd076919e7996ccf', 'request_id': 'req-43bcb251-be6a-4071-8f19-ca2d506d6af5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:54.433828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] The token cache is being invalidate because project a444a4e5ba2d4678bd076919e7996ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:54.434258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:54.434957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-43bcb251-be6a-4071-8f19-ca2d506d6af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:54.436793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 943/1878] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:54 2022] DELETE /identity/v3/projects/a444a4e5ba2d4678bd076919e7996ccf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:54.441680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.442145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.442222 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.469975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:54.471296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:54.549260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 936/1879] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:54.550974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.551264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.551538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.572017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:54.573368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:54.591886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 944/1880] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:54.596114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.605517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7112-9ef4-4c7f-a5b6-8c712e07ae4b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.610381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.611062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.611350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.611629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.612364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:54.612694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.614310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:54.626700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 937/1881] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:54.631745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.648595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57af30e-6153-462e-9ba6-5c767ea71dee None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.653215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.653863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.654136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.654403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.655119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.657182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:54.684670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 945/1882] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:54 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:54.688113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.699008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d50523-3976-4d4e-87c7-2c1d73478bca admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.783252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.784072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.785013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.785593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.786570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.789345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:54.820665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 938/1883] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:54 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:54.824478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.834820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecda02-932f-410f-af2a-32ae04bdc10e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.840767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.840983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.841848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.841848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.843788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.845460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:54.870657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 946/1884] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:54.874745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.896013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164df05-5922-4974-b687-7e18cfb81b39 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.902966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66577c87-09c0-4c8b-b1f1-b94a64adb5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.903748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66577c87-09c0-4c8b-b1f1-b94a64adb5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.904122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66577c87-09c0-4c8b-b1f1-b94a64adb5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.904508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66577c87-09c0-4c8b-b1f1-b94a64adb5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.908917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66577c87-09c0-4c8b-b1f1-b94a64adb5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.908917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66577c87-09c0-4c8b-b1f1-b94a64adb5e1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:54.924685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 939/1885] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:54.926299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:54.937505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fac9bb-186a-44f4-8a06-2e4381e89a3c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:54.943477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:54.944570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:54.944997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:54.945567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] PATH_INFO: `/v3/projects/3463134668744f868894da93f0d93584/users/b1c46495ca694e9fb0d256c6f4abce8d/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:54.976180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3463134668744f868894da93f0d93584, user_id=b1c46495ca694e9fb0d256c6f4abce8d, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:54.981645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:55.017278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 947/1886] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:54 2022] PUT /identity/v3/projects/3463134668744f868894da93f0d93584/users/b1c46495ca694e9fb0d256c6f4abce8d/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:55.028189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:55.028524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:55.028899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:55.066390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] MFA Rules not processed for user `b1c46495ca694e9fb0d256c6f4abce8d`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:55.067660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:55.192440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 940/1887] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:55.204299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:55.206046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:55.272923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905f617-9f06-4709-96d0-1751fefa0368 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:55.276868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f577fb5-849f-4501-b02f-8d95eb686a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:55.277541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f577fb5-849f-4501-b02f-8d95eb686a62 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:55.277828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f577fb5-849f-4501-b02f-8d95eb686a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:55.278166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f577fb5-849f-4501-b02f-8d95eb686a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:55.288702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f577fb5-849f-4501-b02f-8d95eb686a62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:55.290586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f577fb5-849f-4501-b02f-8d95eb686a62 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:55.304068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 948/1888] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:57.691342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:57.692268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:57.754381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e222f-8ec2-4787-bad0-b3321b9eedb7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:57.759545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:57.760382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:57.760694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:57.760970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] PATH_INFO: `/v3/users/8934281a28e24f93a058b4e99738014e` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:57.762868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8934281a28e24f93a058b4e99738014e)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:57.764945 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:57.811742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8934281a28e24f93a058b4e99738014e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:57.819200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8934281a28e24f93a058b4e99738014e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:57.819858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8934281a28e24f93a058b4e99738014e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:57.827658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8934281a28e24f93a058b4e99738014e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:57.828334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8934281a28e24f93a058b4e99738014e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:57.845469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 941/1889] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:57 2022] DELETE /identity/v3/users/8934281a28e24f93a058b4e99738014e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:57.850052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:57.850506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:57.850594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:57.867688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:57.869225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:57.928868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 949/1890] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:57.944273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:57.945834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:58.002262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f4293-c3a7-4218-9e10-8b160081efc6 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:58.006577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:58.007227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:58.007508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:58.007783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:58.017482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:58.018850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:58.028313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 942/1891] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:51:58.547945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:58.550750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:58.571520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa5046-d403-45b9-9f8f-c4828c456f19 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:58.576534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:58.577511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:58.577852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:58.578193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] PATH_INFO: `/v3/projects/8ebb54e66dec4709ad23a05f4d793801` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:58.579450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebb54e66dec4709ad23a05f4d793801)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:58.581574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:58.638344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebb54e66dec4709ad23a05f4d793801', 'request_id': 'req-06f821e6-b47e-4027-89c8-602c607c8e87'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.644166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebb54e66dec4709ad23a05f4d793801', 'request_id': 'req-06f821e6-b47e-4027-89c8-602c607c8e87'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.654345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebb54e66dec4709ad23a05f4d793801', 'request_id': 'req-06f821e6-b47e-4027-89c8-602c607c8e87'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.667138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] The token cache is being invalidate because project 8ebb54e66dec4709ad23a05f4d793801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:58.667488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.668207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.669899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 950/1892] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:58 2022] DELETE /identity/v3/projects/8ebb54e66dec4709ad23a05f4d793801 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:58.674704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:58.686735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2d5c1-0e82-4a16-b692-3850edc8dfa5 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:58.745585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:58.746596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:58.746941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:58.747286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] PATH_INFO: `/v3/users/8327063b27d34d7f9da9aeae6c4bd198` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:58.748572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8327063b27d34d7f9da9aeae6c4bd198)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:58.750594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:58.799379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8327063b27d34d7f9da9aeae6c4bd198'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.806071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8327063b27d34d7f9da9aeae6c4bd198'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.806741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8327063b27d34d7f9da9aeae6c4bd198'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.814196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8327063b27d34d7f9da9aeae6c4bd198'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.814876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b47f9ed9-563e-493c-83b6-6b8e74ad50b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8327063b27d34d7f9da9aeae6c4bd198'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:58.832462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 943/1893] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:51:58 2022] DELETE /identity/v3/users/8327063b27d34d7f9da9aeae6c4bd198 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:59.115744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.117233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:59.130033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f821e6-b47e-4027-89c8-602c607c8e87 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.183832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.184718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.184997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.185265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] PATH_INFO: `/v3/projects/429270ef798f48859eb7acbd41ce89f0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.186413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429270ef798f48859eb7acbd41ce89f0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.188278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.231565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429270ef798f48859eb7acbd41ce89f0', 'request_id': 'req-e29527c7-ba51-44bc-8f6a-f63375ee9407'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:59.236598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429270ef798f48859eb7acbd41ce89f0', 'request_id': 'req-e29527c7-ba51-44bc-8f6a-f63375ee9407'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:59.243517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429270ef798f48859eb7acbd41ce89f0', 'request_id': 'req-e29527c7-ba51-44bc-8f6a-f63375ee9407'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:59.253871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] The token cache is being invalidate because project 429270ef798f48859eb7acbd41ce89f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:51:59.254153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:59.254762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e29527c7-ba51-44bc-8f6a-f63375ee9407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:51:59.256784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 951/1894] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:51:59 2022] DELETE /identity/v3/projects/429270ef798f48859eb7acbd41ce89f0 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:59.261564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4eef-3423-426e-b35a-788b941541fd None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.261850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4eef-3423-426e-b35a-788b941541fd None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.262124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4eef-3423-426e-b35a-788b941541fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.281498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-778c4eef-3423-426e-b35a-788b941541fd None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:59.282686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-778c4eef-3423-426e-b35a-788b941541fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:59.339432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 944/1895] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:59.346930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.347214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.347486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.366862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:59.368047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:59.375857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 952/1896] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:51:59.380133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c4eef-3423-426e-b35a-788b941541fd None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.390998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c4eef-3423-426e-b35a-788b941541fd None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.395668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.396329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.396608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.396913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.397635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:51:59.397939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.399520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.411275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 945/1897] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:51:59.415598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.431260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241df5bc-2364-4bf7-9aec-4baf1107c9f1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.431260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.431260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.431260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.431609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.431609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.433467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.459333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 953/1898] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:59.463502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.473318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a4e9-6de8-4d15-96d7-177f4dbf54a1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.526763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.527431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.527713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.528015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.528735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.530632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.549702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 946/1899] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/users => generated 283 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:51:59.554466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.564898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9bfe1-c1c9-4313-995b-3c41330c3aab admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.570568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.571582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.571582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.571845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.572491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.574115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.589348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 954/1900] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:51:59.590021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.600141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d220e-5794-434a-95e4-3e4a07fcfe64 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.605336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f0637-ae60-4577-ba43-dffd698bfd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.605994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f0637-ae60-4577-ba43-dffd698bfd11 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.614263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f0637-ae60-4577-ba43-dffd698bfd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.614263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f0637-ae60-4577-ba43-dffd698bfd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.614263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f0637-ae60-4577-ba43-dffd698bfd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.614263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f0637-ae60-4577-ba43-dffd698bfd11 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.621251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 947/1901] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:51:59.625171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.637671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fdaf4-4d78-4d8b-9713-97ec2fc9ed88 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.642161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.643022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.643298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.643568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] PATH_INFO: `/v3/projects/64883566256249ebb15eb8b521db2052/users/1eba59f47b594590985d99eb133b7f04/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.660062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64883566256249ebb15eb8b521db2052, user_id=1eba59f47b594590985d99eb133b7f04, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.663319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.677025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 955/1902] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:51:59 2022] PUT /identity/v3/projects/64883566256249ebb15eb8b521db2052/users/1eba59f47b594590985d99eb133b7f04/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:51:59.688247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.688546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.688840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.714371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] MFA Rules not processed for user `1eba59f47b594590985d99eb133b7f04`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:51:59.715630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:51:59.778348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 948/1903] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:51:59.784672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.847107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5b1b6-2040-488d-963f-60fe1c53a5b2 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.855366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.855930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.856212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.856485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.857215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.859116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:51:59.891540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 956/1904] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:51:59.893136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:51:59.905574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707803-f0dd-49f8-9124-f813984625a1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:51:59.982110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:51:59.982783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:51:59.983061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:51:59.983333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:51:59.984047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:51:59.985972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.007735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 949/1905] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:51:59 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:00.011742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:00.022445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7e3b9-f064-4cf2-b9c8-aede13597eb9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:00.028160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:00.028869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.029147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.029419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.030140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:00.031754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.045327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 957/1906] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:00.049689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:00.060609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fc2d7-958f-4b9e-9756-019df9db5eec admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:00.065208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:00.066173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.066481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.066758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] PATH_INFO: `/v3/projects/6b6227ea0d2b40ac9751afbe767dc5c8/users/e2bc129dee8a4a539eddf70ea992a3b7/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.082177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6227ea0d2b40ac9751afbe767dc5c8, user_id=e2bc129dee8a4a539eddf70ea992a3b7, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:00.085259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.098774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 950/1907] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:00 2022] PUT /identity/v3/projects/6b6227ea0d2b40ac9751afbe767dc5c8/users/e2bc129dee8a4a539eddf70ea992a3b7/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:00.102790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:00.113609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ee18-705d-471a-815a-f78a1f800932 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:00.172903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e0a7-6c73-4da5-b397-dd41ff4c88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:00.173573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e0a7-6c73-4da5-b397-dd41ff4c88b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.173855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e0a7-6c73-4da5-b397-dd41ff4c88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.174133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e0a7-6c73-4da5-b397-dd41ff4c88b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.174876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e0a7-6c73-4da5-b397-dd41ff4c88b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:00.176456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e0a7-6c73-4da5-b397-dd41ff4c88b0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.187340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 958/1908] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:00.191338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:00.202046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386b529-fd96-475f-900b-05042dfc91fe admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:00.206545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:00.207405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.207679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.207947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] PATH_INFO: `/v3/projects/6b6227ea0d2b40ac9751afbe767dc5c8/users/e2bc129dee8a4a539eddf70ea992a3b7/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.210089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6227ea0d2b40ac9751afbe767dc5c8, user_id=e2bc129dee8a4a539eddf70ea992a3b7, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:00.213372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.224062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 951/1909] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:00 2022] PUT /identity/v3/projects/6b6227ea0d2b40ac9751afbe767dc5c8/users/e2bc129dee8a4a539eddf70ea992a3b7/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:00.234020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.234307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.234578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.261540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] MFA Rules not processed for user `e2bc129dee8a4a539eddf70ea992a3b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:00.262728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:00.328305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 959/1910] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:00.339482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:00.341229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:00.397016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b04070-84d2-41e6-bdd8-77f1071fe465 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:00.400604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dda28-b67d-4821-945e-331f4e2941f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:00.401303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dda28-b67d-4821-945e-331f4e2941f8 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.401580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dda28-b67d-4821-945e-331f4e2941f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.401850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dda28-b67d-4821-945e-331f4e2941f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.411349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dda28-b67d-4821-945e-331f4e2941f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:00.413324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dda28-b67d-4821-945e-331f4e2941f8 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.422449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 952/1911] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:00.817986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:00.881881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c08f7-4694-4f18-8d41-0b3be8f216cf None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:00.886367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:00.887234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.887519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.887792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] PATH_INFO: `/v3/users/1eba59f47b594590985d99eb133b7f04` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:00.889009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eba59f47b594590985d99eb133b7f04)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:00.890929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:00.943815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eba59f47b594590985d99eb133b7f04'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:00.944483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eba59f47b594590985d99eb133b7f04'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:00.951371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eba59f47b594590985d99eb133b7f04'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:00.964808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eba59f47b594590985d99eb133b7f04'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:00.965526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eba59f47b594590985d99eb133b7f04'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:00.985507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 960/1912] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:00 2022] DELETE /identity/v3/users/1eba59f47b594590985d99eb133b7f04 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:00.990009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:00.990467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:00.990467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:01.016321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:01.017570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:01.101640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 953/1913] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:01.118665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:01.120201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:01.184739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918eee0-12c3-4d07-9080-012880514802 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:01.188792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:01.189514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:01.189791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:01.190062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:01.210167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:01.211876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:01.222765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 961/1914] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:01.496998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:01.498408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:01.511346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609ac19-f94a-4e1d-9234-bcf78ec6e5ac None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:01.516462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:01.517513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:01.517941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:01.518379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] PATH_INFO: `/v3/projects/64883566256249ebb15eb8b521db2052` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:01.519704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64883566256249ebb15eb8b521db2052)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:01.522487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:01.569471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64883566256249ebb15eb8b521db2052', 'request_id': 'req-8518f40e-d49e-4fc7-93a8-c84e595a0053'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.574746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64883566256249ebb15eb8b521db2052', 'request_id': 'req-8518f40e-d49e-4fc7-93a8-c84e595a0053'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.580763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64883566256249ebb15eb8b521db2052', 'request_id': 'req-8518f40e-d49e-4fc7-93a8-c84e595a0053'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.592246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] The token cache is being invalidate because project 64883566256249ebb15eb8b521db2052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:01.592535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.593223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.594791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 954/1915] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:01 2022] DELETE /identity/v3/projects/64883566256249ebb15eb8b521db2052 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:01.598861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:01.608273 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca73219-2a82-46d9-9eb6-85530d74a8d1 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:01.661997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:01.662856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:01.663156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:01.663412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] PATH_INFO: `/v3/users/e2bc129dee8a4a539eddf70ea992a3b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:01.664586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bc129dee8a4a539eddf70ea992a3b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:01.666524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:01.711404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bc129dee8a4a539eddf70ea992a3b7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.712050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bc129dee8a4a539eddf70ea992a3b7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.718561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bc129dee8a4a539eddf70ea992a3b7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.725458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bc129dee8a4a539eddf70ea992a3b7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.726088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fc604f48-cafa-4467-b175-2048390a94d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bc129dee8a4a539eddf70ea992a3b7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:01.743018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 962/1916] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:01 2022] DELETE /identity/v3/users/e2bc129dee8a4a539eddf70ea992a3b7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:02.012219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.013689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:02.036274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518f40e-d49e-4fc7-93a8-c84e595a0053 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.097872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.099649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.099927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.100197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] PATH_INFO: `/v3/projects/6b6227ea0d2b40ac9751afbe767dc5c8` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.101387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6227ea0d2b40ac9751afbe767dc5c8)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.103274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.150680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6227ea0d2b40ac9751afbe767dc5c8', 'request_id': 'req-287a1dbe-3a67-4f97-9178-9138578e57c3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:02.157718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6227ea0d2b40ac9751afbe767dc5c8', 'request_id': 'req-287a1dbe-3a67-4f97-9178-9138578e57c3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:02.166423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6227ea0d2b40ac9751afbe767dc5c8', 'request_id': 'req-287a1dbe-3a67-4f97-9178-9138578e57c3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:02.178817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] The token cache is being invalidate because project 6b6227ea0d2b40ac9751afbe767dc5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:02.179101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:02.179742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-287a1dbe-3a67-4f97-9178-9138578e57c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:02.181342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 955/1917] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:02 2022] DELETE /identity/v3/projects/6b6227ea0d2b40ac9751afbe767dc5c8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:02.186341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.186810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.186906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.204050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:02.205234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:02.269638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 963/1918] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:02.277924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.278211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.278482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.301211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:02.302480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:02.311093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 956/1919] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:02.315277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.325245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a3ac-43c7-4b43-a2fa-295a0a2ef9b8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.334162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.334726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.335000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.335269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.335994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:02.336294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.337931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.352693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 964/1920] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:52:02.354629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.364520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01411719-37fc-42b3-908f-469b1517c7c7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.370428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.371057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.371339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.371728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.372301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.374292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.407721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 957/1921] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:02.415628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.425611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f14859-f3e9-436c-9d4c-7a8f9b6dc806 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.481107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.481778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.482077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.482329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.483049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.484997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.508300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 965/1922] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:02.509898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.520196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9aab-16d1-49b6-a877-a5a62e5fe6bd admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.524881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.525547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.525826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.526097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.526826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.528406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.540711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 958/1923] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:02.543158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.553756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29a03-02d0-4468-8d31-77f0d05eb145 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.558575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491fc81-b882-4880-a5b0-28f57de25de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.559238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491fc81-b882-4880-a5b0-28f57de25de9 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.559522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491fc81-b882-4880-a5b0-28f57de25de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.559808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491fc81-b882-4880-a5b0-28f57de25de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.560529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491fc81-b882-4880-a5b0-28f57de25de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.562143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491fc81-b882-4880-a5b0-28f57de25de9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.574761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 966/1924] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:02.578908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.588768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e076-6e78-4717-8c91-fa6f141b0af2 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.593281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.594153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.594432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.594710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] PATH_INFO: `/v3/projects/7625faac58964e5a84cdfcdedeaea4c4/users/6f9348496539456c80f6a5ccfad5d044/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.608871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7625faac58964e5a84cdfcdedeaea4c4, user_id=6f9348496539456c80f6a5ccfad5d044, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.611623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.625356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 959/1925] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:02 2022] PUT /identity/v3/projects/7625faac58964e5a84cdfcdedeaea4c4/users/6f9348496539456c80f6a5ccfad5d044/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:02.637115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.637402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.637682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.663184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] MFA Rules not processed for user `6f9348496539456c80f6a5ccfad5d044`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:02.664367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:02.727522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 967/1926] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:02.732136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.809393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34822-5326-4785-9a2b-c57d811f549f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.813968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.814618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.814895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.815165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.815855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.817806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.841589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 960/1927] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:02.845599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.856992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1888355-a543-4a3b-b531-586a8b84d81b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.909491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.910158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.910443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.910720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.911953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.913879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.933717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 968/1928] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:02 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:02.937715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.947562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5459-263b-4cc9-af81-e03a984e5920 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.952586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.953310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.953587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.953857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:02.954571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:02.956160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:02.966544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 961/1929] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:02.970614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:02.980955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b645c2-ed73-4737-90de-85f41c50a968 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:02.985486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:02.986346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:02.986623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:02.986895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] PATH_INFO: `/v3/projects/9f34c3c26ca04e2c9b6a3244b11e956d/users/61474cd47d1e4e6c9150d9e966d85190/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:03.001329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f34c3c26ca04e2c9b6a3244b11e956d, user_id=61474cd47d1e4e6c9150d9e966d85190, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:03.003944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:03.016515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 969/1930] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:02 2022] PUT /identity/v3/projects/9f34c3c26ca04e2c9b6a3244b11e956d/users/61474cd47d1e4e6c9150d9e966d85190/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:03.020187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:03.030510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c089f-82e0-4fd9-92c4-c65b5d3344a7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:03.090811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423681a-3f85-45da-a019-e661758c0df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:03.091537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423681a-3f85-45da-a019-e661758c0df0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:03.091825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423681a-3f85-45da-a019-e661758c0df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:03.092095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423681a-3f85-45da-a019-e661758c0df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:03.092902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423681a-3f85-45da-a019-e661758c0df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:03.094873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423681a-3f85-45da-a019-e661758c0df0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:03.108612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 962/1931] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:03.110009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:03.121836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61dbf-f0d9-4ff4-a2cd-5b05dc811beb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:03.126898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:03.127650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:03.127933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:03.128247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] PATH_INFO: `/v3/projects/9f34c3c26ca04e2c9b6a3244b11e956d/users/61474cd47d1e4e6c9150d9e966d85190/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:03.130634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f34c3c26ca04e2c9b6a3244b11e956d, user_id=61474cd47d1e4e6c9150d9e966d85190, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:03.133362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:03.146477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 970/1932] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:03 2022] PUT /identity/v3/projects/9f34c3c26ca04e2c9b6a3244b11e956d/users/61474cd47d1e4e6c9150d9e966d85190/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:03.157075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:03.157374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:03.157645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:03.186326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] MFA Rules not processed for user `61474cd47d1e4e6c9150d9e966d85190`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:03.187659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:03.266724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 963/1933] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:03.286199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:03.288491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:03.344193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ecbc-8e49-4056-8bc3-4b3b10ae275b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:03.347891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:03.348551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:03.348881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:03.349718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:03.358867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:03.360300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:03.369921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 971/1934] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:03.454631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:03.464935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e76dba-da49-4bb7-be60-451af65aecad None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:03.472355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d49774-601a-4bc2-8c79-b6cfdc2fd24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:03.473684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d49774-601a-4bc2-8c79-b6cfdc2fd24a service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:03.474021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d49774-601a-4bc2-8c79-b6cfdc2fd24a service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:03.474302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d49774-601a-4bc2-8c79-b6cfdc2fd24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:03.530314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d49774-601a-4bc2-8c79-b6cfdc2fd24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:03.531739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d49774-601a-4bc2-8c79-b6cfdc2fd24a service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:03.541386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 964/1935] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:27.679227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:27.737285 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a06762-50d7-4dc9-a064-9e896d83673f service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:27.742202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:27.743048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:27.743327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:27.743598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] PATH_INFO: `/v3/users/6f9348496539456c80f6a5ccfad5d044` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:27.744787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9348496539456c80f6a5ccfad5d044)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:27.746683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:27.789979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9348496539456c80f6a5ccfad5d044'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:27.790550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9348496539456c80f6a5ccfad5d044'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:27.797052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9348496539456c80f6a5ccfad5d044'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:27.804564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9348496539456c80f6a5ccfad5d044'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:27.805205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9348496539456c80f6a5ccfad5d044'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:27.821671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 972/1936] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:27 2022] DELETE /identity/v3/users/6f9348496539456c80f6a5ccfad5d044 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:27.826123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:27.826613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:27.826821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:27.843601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:27.844845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:27.902172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 965/1937] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:27.916288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:27.916488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:27.979981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdcf22-7e8c-4ea7-a9e8-3a70ac227ef4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:27.983963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:27.984680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:27.985036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:27.985377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:27.997468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:27.999395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:28.010393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 973/1938] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:28.332052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:28.334058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:28.347787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656affe-11d1-48f2-b4c4-aedd4f674ba2 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:28.353620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083db308-94f5-4505-8830-a867807309dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:28.354521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083db308-94f5-4505-8830-a867807309dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:28.354753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083db308-94f5-4505-8830-a867807309dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:28.355063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083db308-94f5-4505-8830-a867807309dd admin admin] PATH_INFO: `/v3/projects/7625faac58964e5a84cdfcdedeaea4c4` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:28.356259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083db308-94f5-4505-8830-a867807309dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7625faac58964e5a84cdfcdedeaea4c4)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:28.358382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083db308-94f5-4505-8830-a867807309dd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:28.410551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7625faac58964e5a84cdfcdedeaea4c4', 'request_id': 'req-083db308-94f5-4505-8830-a867807309dd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.417549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7625faac58964e5a84cdfcdedeaea4c4', 'request_id': 'req-083db308-94f5-4505-8830-a867807309dd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.426455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7625faac58964e5a84cdfcdedeaea4c4', 'request_id': 'req-083db308-94f5-4505-8830-a867807309dd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.441624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-083db308-94f5-4505-8830-a867807309dd admin admin] The token cache is being invalidate because project 7625faac58964e5a84cdfcdedeaea4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:28.442084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.442920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.445049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 966/1939] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:28 2022] DELETE /identity/v3/projects/7625faac58964e5a84cdfcdedeaea4c4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:28.449305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:28.461023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959af0d-ab39-4e9b-a961-25fa06484ef9 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:28.531540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:28.532416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:28.532726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:28.533005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] PATH_INFO: `/v3/users/61474cd47d1e4e6c9150d9e966d85190` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:28.534448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61474cd47d1e4e6c9150d9e966d85190)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:28.536387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:28.581006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61474cd47d1e4e6c9150d9e966d85190'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.582013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61474cd47d1e4e6c9150d9e966d85190'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.589627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61474cd47d1e4e6c9150d9e966d85190'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.598369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61474cd47d1e4e6c9150d9e966d85190'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.599077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-89b05909-fed2-476f-a945-4bdef7627e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61474cd47d1e4e6c9150d9e966d85190'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:28.615928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 974/1940] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:28 2022] DELETE /identity/v3/users/61474cd47d1e4e6c9150d9e966d85190 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:28.902516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:28.904194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:28.918529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083db308-94f5-4505-8830-a867807309dd admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:28.977788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:28.978646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:28.978923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:28.979192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] PATH_INFO: `/v3/projects/9f34c3c26ca04e2c9b6a3244b11e956d` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:28.980772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f34c3c26ca04e2c9b6a3244b11e956d)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:28.982689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.029176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f34c3c26ca04e2c9b6a3244b11e956d', 'request_id': 'req-c49a298f-eb85-472f-b5bb-a8147484b628'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.040926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f34c3c26ca04e2c9b6a3244b11e956d', 'request_id': 'req-c49a298f-eb85-472f-b5bb-a8147484b628'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.047186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f34c3c26ca04e2c9b6a3244b11e956d', 'request_id': 'req-c49a298f-eb85-472f-b5bb-a8147484b628'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.060868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] The token cache is being invalidate because project 9f34c3c26ca04e2c9b6a3244b11e956d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:29.061147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.061763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c49a298f-eb85-472f-b5bb-a8147484b628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.063582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 967/1941] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:28 2022] DELETE /identity/v3/projects/9f34c3c26ca04e2c9b6a3244b11e956d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:29.069119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.069647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.069766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.087356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:29.088566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.150668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 975/1942] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:29.162354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.162788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.163300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.183324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:29.184519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.193213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 968/1943] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:29.197443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.206910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23564b1f-1412-4318-8164-3e7a4e8f8b78 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.211555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.212280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.212555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.212861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.213589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:29.213893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.215523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.228690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 976/1944] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:52:29.232460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.242320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af67a-53c2-4429-b96f-31d737141b58 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.247016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.247669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.247981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.248258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.248991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.250927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.278332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 969/1945] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:29.282357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.292768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf67cc-cecd-4e2e-bc7a-594c8c6f1a19 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.353130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.353728 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.354014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.354293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.355158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.357116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.383526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 977/1946] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:29.385063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.398505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc61b-5604-45df-81b7-da9ddfe9d9c0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.406281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.407379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.407768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.408147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.409662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.412013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.426442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 970/1947] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:29.428325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.440363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118110-51d9-4f97-bb0a-af920e8a5c55 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.445045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.445704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.445983 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.446254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.446967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.448535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.460448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 978/1948] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:29.464465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.481253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156384f9-70bc-46b2-b1a1-e13477195c68 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.497271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.497271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.497643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee43c52-4ebb-45d3-90e8-9fba9df1e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.500924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee43c52-4ebb-45d3-90e8-9fba9df1e1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.501016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee43c52-4ebb-45d3-90e8-9fba9df1e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.501488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee43c52-4ebb-45d3-90e8-9fba9df1e1a4 admin admin] PATH_INFO: `/v3/projects/6c97cb7ba2144267a38d0d0f48201172/users/25fb4a3a1c244676b62dde667e692b11/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.512839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3b55-d456-43de-ae5d-a307946b27bf admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.518198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.519066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.519338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.519764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] PATH_INFO: `/v3/users/766f124af4444076a76faa6049c2ddfd` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.521302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766f124af4444076a76faa6049c2ddfd)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.523440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.524336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee43c52-4ebb-45d3-90e8-9fba9df1e1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c97cb7ba2144267a38d0d0f48201172, user_id=25fb4a3a1c244676b62dde667e692b11, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.528253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee43c52-4ebb-45d3-90e8-9fba9df1e1a4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.551642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 971/1949] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:29 2022] PUT /identity/v3/projects/6c97cb7ba2144267a38d0d0f48201172/users/25fb4a3a1c244676b62dde667e692b11/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:29.562342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.562796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.563209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.592912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] MFA Rules not processed for user `25fb4a3a1c244676b62dde667e692b11`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:29.594115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.596935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766f124af4444076a76faa6049c2ddfd'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.598235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766f124af4444076a76faa6049c2ddfd'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.613950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766f124af4444076a76faa6049c2ddfd'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.625123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766f124af4444076a76faa6049c2ddfd'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.626394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9b94cc9a-dfcf-490d-8c36-8fe5f16677d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766f124af4444076a76faa6049c2ddfd'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:29.648529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 979/1950] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:29 2022] DELETE /identity/v3/users/766f124af4444076a76faa6049c2ddfd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:29.658281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.658664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.659278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.678095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:29.679429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.689162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 972/1951] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:29.694877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.695473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.760804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 980/1952] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:29.771877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.773431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:29.779644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f43128-ec6c-4160-80a5-421e1bda046e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.785472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.786131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.787414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.787414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.787982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.790070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.819740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 973/1953] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:29.821026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.844100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7bb53-6d62-49a5-b0c4-f03684e10830 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.879614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353540f-61de-433c-8943-ca28b91a6dc3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.884108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.884796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.885075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.885343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.886022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.887921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.891488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.892673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.892673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.892941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.904755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.906455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.918725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 981/1954] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:52:29.930552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:29.932716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 974/1955] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:29 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:29.941849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae886f-4aff-4c15-9aeb-610894010b0b service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:29.952019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:29.952939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:29.953322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:29.953777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:29.957197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:29.972945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:29.999821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 982/1956] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:30.001276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.035804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f8e76-e93a-45f4-aefd-4ddfbca01b07 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.047590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.049937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.050289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.050618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] PATH_INFO: `/v3/projects/f54feb84dd3a41728cad97310722f76a/users/f00c01bbaf6f4ce0835f9d77809c782c/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.080810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54feb84dd3a41728cad97310722f76a, user_id=f00c01bbaf6f4ce0835f9d77809c782c, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:30.085547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:30.108684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 975/1957] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:30 2022] PUT /identity/v3/projects/f54feb84dd3a41728cad97310722f76a/users/f00c01bbaf6f4ce0835f9d77809c782c/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:30.110576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.124989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62802b-bc2a-41d8-a8fa-e84f826b2096 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.196264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c9863-628a-47c1-ae34-7d3496f537d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.197076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9863-628a-47c1-ae34-7d3496f537d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.197218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9863-628a-47c1-ae34-7d3496f537d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.197489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c9863-628a-47c1-ae34-7d3496f537d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.198292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c9863-628a-47c1-ae34-7d3496f537d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:30.199838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c9863-628a-47c1-ae34-7d3496f537d6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:30.215106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 983/1958] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:30.216074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.232184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44031df7-93b5-482d-8a16-dfd5846b2479 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.237205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.238464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.238464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.238919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] PATH_INFO: `/v3/projects/f54feb84dd3a41728cad97310722f76a/users/f00c01bbaf6f4ce0835f9d77809c782c/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.241595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54feb84dd3a41728cad97310722f76a, user_id=f00c01bbaf6f4ce0835f9d77809c782c, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:30.244012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:30.255699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 976/1959] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:30 2022] PUT /identity/v3/projects/f54feb84dd3a41728cad97310722f76a/users/f00c01bbaf6f4ce0835f9d77809c782c/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:30.265898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.266188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.266460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.292990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] MFA Rules not processed for user `f00c01bbaf6f4ce0835f9d77809c782c`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:30.294228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:30.383744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 984/1960] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:30.398499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.400093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:30.456822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895a454-561b-4e41-b900-9184a961de32 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.460824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.461494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.461776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.462051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.471934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:30.473529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:30.483965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 977/1961] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:30.513259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.588918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d70c3-2a43-4b21-9260-f94d0ca0a4c6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.593800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.595120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.595512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.595889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] PATH_INFO: `/v3/users/25fb4a3a1c244676b62dde667e692b11` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.597411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] RBAC: Authorizing `identity:delete_user(user_id=25fb4a3a1c244676b62dde667e692b11)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:30.599633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:30.651892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25fb4a3a1c244676b62dde667e692b11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.652601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25fb4a3a1c244676b62dde667e692b11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.664144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25fb4a3a1c244676b62dde667e692b11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.673599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25fb4a3a1c244676b62dde667e692b11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.674220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-495ec3ef-061b-44a6-bde0-5e26cc83323d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25fb4a3a1c244676b62dde667e692b11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.688189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.689856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:30.693293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 985/1962] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:30 2022] DELETE /identity/v3/users/25fb4a3a1c244676b62dde667e692b11 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:30.697797 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fff48-8a51-40e8-9093-1562529df466 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.698082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fff48-8a51-40e8-9093-1562529df466 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.698411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fff48-8a51-40e8-9093-1562529df466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.716352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-133fff48-8a51-40e8-9093-1562529df466 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:30.717464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c274a-ca9b-461b-9e6a-96f5848e1dc0 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.717615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-133fff48-8a51-40e8-9093-1562529df466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:30.780457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 986/1963] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:30.785851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.787593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.787882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.788209 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] PATH_INFO: `/v3/projects/7a1f37f5754b44f5a2f2a85b45442782` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.789695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1f37f5754b44f5a2f2a85b45442782)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:30.791201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fff48-8a51-40e8-9093-1562529df466 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:30.791662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:30.792690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-133fff48-8a51-40e8-9093-1562529df466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:30.847502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1f37f5754b44f5a2f2a85b45442782', 'request_id': 'req-b2719416-0bda-4c9b-a4de-b83ee6e23599'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.858400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1f37f5754b44f5a2f2a85b45442782', 'request_id': 'req-b2719416-0bda-4c9b-a4de-b83ee6e23599'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.866275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fff48-8a51-40e8-9093-1562529df466 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:30.867740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1f37f5754b44f5a2f2a85b45442782', 'request_id': 'req-b2719416-0bda-4c9b-a4de-b83ee6e23599'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.899427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] The token cache is being invalidate because project 7a1f37f5754b44f5a2f2a85b45442782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:30.900070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.901181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b2719416-0bda-4c9b-a4de-b83ee6e23599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:30.908688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 978/1964] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:30 2022] DELETE /identity/v3/projects/7a1f37f5754b44f5a2f2a85b45442782 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:30.908688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5121e5c-f7c7-46f2-bb0a-c14ffcce3e18 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.908895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5121e5c-f7c7-46f2-bb0a-c14ffcce3e18 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.914891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5121e5c-f7c7-46f2-bb0a-c14ffcce3e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.929858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:30.930686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:30.931096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:30.931523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:30.934080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c5121e5c-f7c7-46f2-bb0a-c14ffcce3e18 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:30.934546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:30.935251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c5121e5c-f7c7-46f2-bb0a-c14ffcce3e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.000541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.002009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.011417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 987/1965] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:52:31.012216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.013488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.016104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 979/1966] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:31.023469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.023757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.024038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.024800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9e46-bd8b-40c6-b5a9-078adac59d3f service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.029637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.030572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.030911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.031503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] PATH_INFO: `/v3/users/b1c46495ca694e9fb0d256c6f4abce8d` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.032511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c46495ca694e9fb0d256c6f4abce8d)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.034755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.044759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:31.046514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.060371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 980/1967] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:31.062320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.072798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c91bb-788b-4331-bc1b-929f31bd02f3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.081222 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.082810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.083448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.083448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.086420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:31.086839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.089560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.098400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c46495ca694e9fb0d256c6f4abce8d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.099574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c46495ca694e9fb0d256c6f4abce8d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.103254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 981/1968] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:52:31.108352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.108563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c46495ca694e9fb0d256c6f4abce8d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.119467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c46495ca694e9fb0d256c6f4abce8d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.120129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-95698dd7-0089-4b08-a684-f38e7b19dc0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c46495ca694e9fb0d256c6f4abce8d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.123871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64d8d5-0558-4b0d-8a40-c5a4c5f87578 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.129527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.130892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.131426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.131986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.133095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.136312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.140346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 988/1969] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:31 2022] DELETE /identity/v3/users/b1c46495ca694e9fb0d256c6f4abce8d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:31.145054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.145400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.146672 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.171784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:31.173794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 982/1970] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:31 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:31.177122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.177634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.179476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.264807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba9c47-3cda-49ee-933a-fef59a5318ed admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.269841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 989/1971] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:31.270922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.271893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.277875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.277875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.277875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.279843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.281892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.282725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.314936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 983/1972] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:31 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:31.319732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.332381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863219-f2ac-4adc-89dd-17a379fa7f1d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.339314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.340094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.340582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.341314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.345112 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.352987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653f427-0b02-46b1-b979-79ac1e3fd0df None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.353653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.357031 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.357828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.358293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.358997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.370970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 984/1973] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:31.374590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.377675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.380798 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.389121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76a172-2082-40b5-83ab-4e1356067cac admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.398405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.399166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.399524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.400007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.400853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.402837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.405819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 990/1974] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:31.421641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 985/1975] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:31.421974 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.423648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.425035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.446586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb61627-2994-4fa2-830d-464f8239d23b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.457222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061ad5c-7357-439c-9d6b-3d43e14ccd19 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.459646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c9235-788b-495b-a50d-db01b3c8dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.461946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c9235-788b-495b-a50d-db01b3c8dfce admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.462235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c9235-788b-495b-a50d-db01b3c8dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.462515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c9235-788b-495b-a50d-db01b3c8dfce admin admin] PATH_INFO: `/v3/projects/3bd8178cdc924ded873c5f7b7fba2e48/users/8d0e7a7b5f3a420aa1faf9d88bd8678b/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.470201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.472145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.473009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.475242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] PATH_INFO: `/v3/projects/6c97cb7ba2144267a38d0d0f48201172` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.477715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c97cb7ba2144267a38d0d0f48201172)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.480713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.483079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c9235-788b-495b-a50d-db01b3c8dfce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd8178cdc924ded873c5f7b7fba2e48, user_id=8d0e7a7b5f3a420aa1faf9d88bd8678b, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.487419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c9235-788b-495b-a50d-db01b3c8dfce admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.521015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 986/1976] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:31 2022] PUT /identity/v3/projects/3bd8178cdc924ded873c5f7b7fba2e48/users/8d0e7a7b5f3a420aa1faf9d88bd8678b/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:31.523312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.523814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.524243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.582095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] MFA Rules not processed for user `8d0e7a7b5f3a420aa1faf9d88bd8678b`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:31.583843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.591211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c97cb7ba2144267a38d0d0f48201172', 'request_id': 'req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.604218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c97cb7ba2144267a38d0d0f48201172', 'request_id': 'req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.623415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c97cb7ba2144267a38d0d0f48201172', 'request_id': 'req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.653708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] The token cache is being invalidate because project 6c97cb7ba2144267a38d0d0f48201172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:31.656156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.658188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.662523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 991/1977] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:31 2022] DELETE /identity/v3/projects/6c97cb7ba2144267a38d0d0f48201172 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:31.665933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.674113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.732614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 987/1978] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:31.747926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:31.750095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:31.772500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8554cce-6c12-49af-bf60-bf8ad3cf9c72 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.794888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.798137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.798591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.800183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] PATH_INFO: `/v3/users/f00c01bbaf6f4ce0835f9d77809c782c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.802674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00c01bbaf6f4ce0835f9d77809c782c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.805078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.842481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5b54e-9a51-4343-9a68-f39b726294d0 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:31.849232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:31.862660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:31.862660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:31.862962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:31.874503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:31.875953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:31.882096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00c01bbaf6f4ce0835f9d77809c782c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.883073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00c01bbaf6f4ce0835f9d77809c782c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.888689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 988/1979] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:31.918618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00c01bbaf6f4ce0835f9d77809c782c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.935501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00c01bbaf6f4ce0835f9d77809c782c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.937387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00c01bbaf6f4ce0835f9d77809c782c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:31.967691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 992/1980] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:31 2022] DELETE /identity/v3/users/f00c01bbaf6f4ce0835f9d77809c782c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:32.105821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.108425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.148260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885b5b6-8147-4219-87a2-acbebdb719ba service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.240104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.241384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.241738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.242119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] PATH_INFO: `/v3/projects/3463134668744f868894da93f0d93584` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.243436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3463134668744f868894da93f0d93584)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.245972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.322681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3463134668744f868894da93f0d93584', 'request_id': 'req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.328984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3463134668744f868894da93f0d93584', 'request_id': 'req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.336727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3463134668744f868894da93f0d93584', 'request_id': 'req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.350853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] The token cache is being invalidate because project 3463134668744f868894da93f0d93584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:32.351146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.351784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b03cd596-3e7f-4ab6-8c83-13fe616d70aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.355455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 989/1981] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:32 2022] DELETE /identity/v3/projects/3463134668744f868894da93f0d93584 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:32.357557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.359109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.359956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f5ac-7376-4f40-863c-879b36a987e0 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.360248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f5ac-7376-4f40-863c-879b36a987e0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.360534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f5ac-7376-4f40-863c-879b36a987e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.384389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-e981f5ac-7376-4f40-863c-879b36a987e0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:32.385781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e981f5ac-7376-4f40-863c-879b36a987e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.391359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701228-95d9-46ae-9e45-d4b8131fadd6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.443783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 990/1982] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:32.447179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.448039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.448320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.448594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] PATH_INFO: `/v3/projects/f54feb84dd3a41728cad97310722f76a` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.449891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54feb84dd3a41728cad97310722f76a)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.451452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.451740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.451844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.452092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.474231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:32.475622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.486296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 991/1983] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:32.490250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.499719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d6aaf-388a-4c31-8bb4-8b901eab96a3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.515203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54feb84dd3a41728cad97310722f76a', 'request_id': 'req-75a6d6c6-f531-4505-804a-2d99e7360a5d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.521300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54feb84dd3a41728cad97310722f76a', 'request_id': 'req-75a6d6c6-f531-4505-804a-2d99e7360a5d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.529054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54feb84dd3a41728cad97310722f76a', 'request_id': 'req-75a6d6c6-f531-4505-804a-2d99e7360a5d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.540889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] The token cache is being invalidate because project f54feb84dd3a41728cad97310722f76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:32.541177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.541801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-75a6d6c6-f531-4505-804a-2d99e7360a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:32.543331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 993/1984] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:32 2022] DELETE /identity/v3/projects/f54feb84dd3a41728cad97310722f76a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:32.548985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b47a5-13c2-4974-a789-4239e3e43d85 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.549498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b47a5-13c2-4974-a789-4239e3e43d85 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.550342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b47a5-13c2-4974-a789-4239e3e43d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.561678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.562340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.562619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.562891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.563799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:32.564102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.565726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.573278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9b6b47a5-13c2-4974-a789-4239e3e43d85 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:32.574454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9b6b47a5-13c2-4974-a789-4239e3e43d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.577601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 992/1985] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:52:32.581853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.582277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 994/1986] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:32.584283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.591277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.591566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.591838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.596378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a3731-b230-42f0-a9f7-10fc52d22d1b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.600968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.601615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.601894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.602165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.602839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.604796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.609645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:32.610820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:32.619037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 995/1987] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:32.623944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.633888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 993/1988] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:32.634174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af00719-dd50-4a21-a48b-4ca00941e1ec None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.638059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.698658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.699407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.699690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.699987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.700758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:32.701065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.702651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.715663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 996/1989] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:52:32.719672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa0487-6407-42e9-bc4a-861edda11555 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.719893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.727381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.728391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.728767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.729116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.729862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.730072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569ef89-ead5-4ec6-b2de-e5981aae5bdf admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.732102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.734754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.735388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.735667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.735936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.736617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.738601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.757874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 994/1990] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:32.761656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.774546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3053-b28d-4ca7-a966-16285214e7ba admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.775292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 997/1991] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:32.779080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.844736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.845751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.845901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.846085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.846400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae13c3-2e40-4de6-848c-65b4458c4876 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.846817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.848528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.851308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.851965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.852243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.852513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.853224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.855142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.861771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 995/1992] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:32.862603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.873670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5dc94-a082-4c15-9af4-f35d4802e642 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.884699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 998/1993] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:32.887740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.888449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.888760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.889075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.889203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.889800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.891386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.902844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 996/1994] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:32.907414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.909389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f29f6-78a4-4366-958a-fe3ae0223d78 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.916231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.917177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.917635 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.918037 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.918176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd06fd5-42a0-416c-9707-f60131d0f924 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.919056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.921312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.923130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1f4bf-a757-4051-ada4-6498f05badba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.924028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1f4bf-a757-4051-ada4-6498f05badba admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.924305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1f4bf-a757-4051-ada4-6498f05badba admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.924947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1f4bf-a757-4051-ada4-6498f05badba admin admin] PATH_INFO: `/v3/projects/0e91eab6ef81473c97f4c7dfd04506f5/users/18a2a873aeae41de8b25c2095c645479/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.937653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 999/1995] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:32.939133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.943192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1f4bf-a757-4051-ada4-6498f05badba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e91eab6ef81473c97f4c7dfd04506f5, user_id=18a2a873aeae41de8b25c2095c645479, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.946070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1f4bf-a757-4051-ada4-6498f05badba admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.951640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ee23-08f6-4499-870d-a8acd4558e4a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:32.958274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:32.959178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.959564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.959904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.960726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:32.962454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:32.966184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 997/1996] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:32 2022] PUT /identity/v3/projects/0e91eab6ef81473c97f4c7dfd04506f5/users/18a2a873aeae41de8b25c2095c645479/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:32.970896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:32.971241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:32.971586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:32.985375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1000/1997] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:32.990958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:32.995700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:32.996930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:33.008400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5dbe0-ed3d-4206-ad6c-561239c13b38 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.083902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 998/1998] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:33.088613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d829c-96b2-4a30-afc2-1022ca900ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.090108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d829c-96b2-4a30-afc2-1022ca900ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.090108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d829c-96b2-4a30-afc2-1022ca900ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.090108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d829c-96b2-4a30-afc2-1022ca900ddf admin admin] PATH_INFO: `/v3/projects/6d58e843a1554c66b471f252efb8984a/users/c9e9ea7f4a264f089950ccb7a4156101/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.105806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d829c-96b2-4a30-afc2-1022ca900ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d58e843a1554c66b471f252efb8984a, user_id=c9e9ea7f4a264f089950ccb7a4156101, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.108457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d829c-96b2-4a30-afc2-1022ca900ddf admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.113128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.115865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:33.126346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1001/1999] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:32 2022] PUT /identity/v3/projects/6d58e843a1554c66b471f252efb8984a/users/c9e9ea7f4a264f089950ccb7a4156101/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:33.138433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.138999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.139553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.178800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] MFA Rules not processed for user `c9e9ea7f4a264f089950ccb7a4156101`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:33.180660 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:33.199304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58024a2c-2878-4227-89d2-22b73ba405a4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.264403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1002/2000] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:33.265447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.269629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.270782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.271712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.272295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.366164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.368095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.379530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 999/2001] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:52:33.407536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e83a2-52c1-4372-8085-5bd13f72f5c0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.421115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.422154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.422788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.423847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.424701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.430144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.497510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1003/2002] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:33 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:33.499744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.514656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb47914-1e96-45d2-a272-a4946b1465ac service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.619221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.619912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.620138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.620422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.621128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.623068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.655825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1000/2003] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:33 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:33.661177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.681276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd36ee-d301-4871-9fd4-75d2d14086e3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.688954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.689754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.690039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.690316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.691053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.692891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.707792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1004/2004] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:33.709612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.719359 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213a08a-0059-4a0f-b89b-7f5fde691a55 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.737643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.738695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.739072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.739629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] PATH_INFO: `/v3/projects/f539094817734f738ce7838ce7787d23/users/e13411e363ab45d39bf6ff96d0896451/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.757504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f539094817734f738ce7838ce7787d23, user_id=e13411e363ab45d39bf6ff96d0896451, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.760287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.780983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1001/2005] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:33 2022] PUT /identity/v3/projects/f539094817734f738ce7838ce7787d23/users/e13411e363ab45d39bf6ff96d0896451/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:33.785022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.795097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6209d42-f7f0-44e4-9fbf-816ed24e4ca9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.855100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1536246-b1f2-4af0-86c0-e8b95b63267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.855768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536246-b1f2-4af0-86c0-e8b95b63267c admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.856053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536246-b1f2-4af0-86c0-e8b95b63267c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.856929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536246-b1f2-4af0-86c0-e8b95b63267c admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.857778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1536246-b1f2-4af0-86c0-e8b95b63267c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.859464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1536246-b1f2-4af0-86c0-e8b95b63267c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.870047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1005/2006] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:33.873901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:33.884895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3b10f-6cc5-4212-b836-120c549e9ec9 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:33.891087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:33.892280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.892776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.893254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] PATH_INFO: `/v3/projects/f539094817734f738ce7838ce7787d23/users/e13411e363ab45d39bf6ff96d0896451/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.896121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f539094817734f738ce7838ce7787d23, user_id=e13411e363ab45d39bf6ff96d0896451, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:33.899187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:33.916549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1002/2007] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:33 2022] PUT /identity/v3/projects/f539094817734f738ce7838ce7787d23/users/e13411e363ab45d39bf6ff96d0896451/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:33.927123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:33.927420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:33.927686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:33.954361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] MFA Rules not processed for user `e13411e363ab45d39bf6ff96d0896451`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:33.961638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:34.057553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1006/2008] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:34.070244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:34.072239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:34.133518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11baae4-e328-41ab-bf8e-c85f4bedb9f5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:34.137276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba4a8-b2f1-4365-bad7-49a868ff7231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:34.137923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba4a8-b2f1-4365-bad7-49a868ff7231 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:34.138202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba4a8-b2f1-4365-bad7-49a868ff7231 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:34.138472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba4a8-b2f1-4365-bad7-49a868ff7231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:34.147658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ba4a8-b2f1-4365-bad7-49a868ff7231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:34.149152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ba4a8-b2f1-4365-bad7-49a868ff7231 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:34.159334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1003/2009] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:34.359724 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:34.443264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0631c9-3802-4eff-a96b-d7c5e2967b7a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:34.448001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:34.448906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:34.449317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:34.449427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] PATH_INFO: `/v3/users/c9e9ea7f4a264f089950ccb7a4156101` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:34.450655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e9ea7f4a264f089950ccb7a4156101)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:34.452799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:34.523442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e9ea7f4a264f089950ccb7a4156101'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:34.524122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e9ea7f4a264f089950ccb7a4156101'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:34.532898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e9ea7f4a264f089950ccb7a4156101'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:34.554433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e9ea7f4a264f089950ccb7a4156101'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:34.556016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e9ea7f4a264f089950ccb7a4156101'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:34.611756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1007/2010] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:34 2022] DELETE /identity/v3/users/c9e9ea7f4a264f089950ccb7a4156101 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:34.615179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:34.615705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:34.616052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:34.657045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:34.664589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:34.840204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1004/2011] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:34.850251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:34.852492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:34.952060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffab3df-c7a9-4e82-8633-ea78d282f334 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:34.956135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:34.962389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:34.962389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:34.962389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:34.968241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:34.970023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:34.983295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1008/2012] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:35.431975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:35.434081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:35.453516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4512e-0fc4-46a8-a55b-6c99cd2906b6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:35.462572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:35.465386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:35.467187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:35.467673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] PATH_INFO: `/v3/projects/6d58e843a1554c66b471f252efb8984a` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:35.469723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d58e843a1554c66b471f252efb8984a)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:35.472947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:35.549826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d58e843a1554c66b471f252efb8984a', 'request_id': 'req-4d43e39b-93f3-4044-9a32-7c9bfc096a06'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.556813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d58e843a1554c66b471f252efb8984a', 'request_id': 'req-4d43e39b-93f3-4044-9a32-7c9bfc096a06'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.563579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d58e843a1554c66b471f252efb8984a', 'request_id': 'req-4d43e39b-93f3-4044-9a32-7c9bfc096a06'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.580211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] The token cache is being invalidate because project 6d58e843a1554c66b471f252efb8984a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:35.580563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.581334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.583033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1005/2013] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:35 2022] DELETE /identity/v3/projects/6d58e843a1554c66b471f252efb8984a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:35.587036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:35.597586 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c01bb-39db-4efe-9201-73ee0d9b3fc8 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:35.723247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:35.726380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:35.726866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:35.731717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] PATH_INFO: `/v3/users/e13411e363ab45d39bf6ff96d0896451` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:35.735503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13411e363ab45d39bf6ff96d0896451)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:35.737478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:35.866198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13411e363ab45d39bf6ff96d0896451'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.867787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13411e363ab45d39bf6ff96d0896451'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.876390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13411e363ab45d39bf6ff96d0896451'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.890066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13411e363ab45d39bf6ff96d0896451'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.891781 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2338a7c4-da81-4e75-8af2-3d6a0bdfdbed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13411e363ab45d39bf6ff96d0896451'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:35.924028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1009/2014] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:35 2022] DELETE /identity/v3/users/e13411e363ab45d39bf6ff96d0896451 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:36.566071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:36.568179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:36.591063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43e39b-93f3-4044-9a32-7c9bfc096a06 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:36.668535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:36.669982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:36.670475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:36.670826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] PATH_INFO: `/v3/projects/f539094817734f738ce7838ce7787d23` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:36.672446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f539094817734f738ce7838ce7787d23)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:36.676236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:36.728202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f539094817734f738ce7838ce7787d23', 'request_id': 'req-3ea34aaf-6d1c-4850-89b8-1686726aeb36'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:36.734393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f539094817734f738ce7838ce7787d23', 'request_id': 'req-3ea34aaf-6d1c-4850-89b8-1686726aeb36'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:36.741601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f539094817734f738ce7838ce7787d23', 'request_id': 'req-3ea34aaf-6d1c-4850-89b8-1686726aeb36'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:36.756332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] The token cache is being invalidate because project f539094817734f738ce7838ce7787d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:36.756828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:36.757485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-3ea34aaf-6d1c-4850-89b8-1686726aeb36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:36.759595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1006/2015] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:36 2022] DELETE /identity/v3/projects/f539094817734f738ce7838ce7787d23 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:36.763956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:36.764367 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:36.764465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:36.783557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:36.785309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:36.928515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1010/2016] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:36.933292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:36.934344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:36.934879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.041110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:37.050092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:37.080879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1007/2017] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:37.086510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.099985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d83c55-2499-414e-8d67-eb9bf4403306 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.111160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.119326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.119711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.119999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.120845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:37.121059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.124383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.156585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1011/2018] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:52:37.158290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.170796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc96a56-8010-4e40-aa3f-e339f383a1c6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.181509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.182136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.182645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.182759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.183494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.190191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.239136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1008/2019] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:37 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:37.240732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.253012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b973c24-f11d-495a-a08f-576a70b70be6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.329129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.330115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.330490 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.330865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.331628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.334214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.356611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1012/2020] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:37 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:37.361513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.373429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c9ee2-5219-43bc-acf7-cca5c2cffb84 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.380011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.381040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.381334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.381898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.382636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.385030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.403751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1009/2021] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:37.407411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.418615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8fba7-26e7-4f7f-ae80-f7654040be5f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.423574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ec05-c4f4-41cb-9ae7-116ec3ddecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.424523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ec05-c4f4-41cb-9ae7-116ec3ddecd5 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.424665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ec05-c4f4-41cb-9ae7-116ec3ddecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.425041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ec05-c4f4-41cb-9ae7-116ec3ddecd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.425857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775ec05-c4f4-41cb-9ae7-116ec3ddecd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.427730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775ec05-c4f4-41cb-9ae7-116ec3ddecd5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.454087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.458782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1013/2022] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:37.483167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7b16-d087-4feb-b6f2-6b8c6723308e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.491240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.492273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.492701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.493073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] PATH_INFO: `/v3/projects/b734bb722ffd4dd49fb449936221e5e5/users/18e2bfa909334e87bb742767a4e2bd77/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.511744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b734bb722ffd4dd49fb449936221e5e5, user_id=18e2bfa909334e87bb742767a4e2bd77, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.515039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.536685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1010/2023] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:37 2022] PUT /identity/v3/projects/b734bb722ffd4dd49fb449936221e5e5/users/18e2bfa909334e87bb742767a4e2bd77/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:37.543073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.543352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.543630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.611301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] MFA Rules not processed for user `18e2bfa909334e87bb742767a4e2bd77`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:37.612263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:37.709934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1014/2024] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:37.714318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.803669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31014b-14fc-4e1d-80e0-eee2710e19ed admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.819662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.820318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.821193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.821484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.822487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.828853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.858150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1011/2025] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:37 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:37.862176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.874522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f24fe1-cf76-4563-be04-af87af970caf None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.940274 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf252b03-9378-4119-933b-239603002154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.941090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf252b03-9378-4119-933b-239603002154 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.941473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf252b03-9378-4119-933b-239603002154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.941865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf252b03-9378-4119-933b-239603002154 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.943815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf252b03-9378-4119-933b-239603002154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.946511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf252b03-9378-4119-933b-239603002154 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:37.972537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1015/2026] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:37 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:37.976646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:37.989788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b468e-c3f0-4ab5-a36d-24038657b48c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:37.995207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:37.995985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:37.996365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:37.997450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:37.997601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:37.999507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:38.017717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1012/2027] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:38.021965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf252b03-9378-4119-933b-239603002154 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:38.035488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf252b03-9378-4119-933b-239603002154 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:38.040553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:38.041470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:38.041755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:38.042029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] PATH_INFO: `/v3/projects/f4ec509d9b6f4a3392a61ce83282f71a/users/6fa1ada31f08492a8677107d35c2aae0/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:38.068849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ec509d9b6f4a3392a61ce83282f71a, user_id=6fa1ada31f08492a8677107d35c2aae0, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:38.071756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:38.096400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1016/2028] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:38 2022] PUT /identity/v3/projects/f4ec509d9b6f4a3392a61ce83282f71a/users/6fa1ada31f08492a8677107d35c2aae0/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:38.100887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:38.112089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac248e4a-3b74-4bef-a51a-2398c0c40b29 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:38.216305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c120da-fefa-4226-a39f-8476073c185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:38.217178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c120da-fefa-4226-a39f-8476073c185f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:38.218084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c120da-fefa-4226-a39f-8476073c185f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:38.218084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c120da-fefa-4226-a39f-8476073c185f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:38.218540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c120da-fefa-4226-a39f-8476073c185f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:38.220596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c120da-fefa-4226-a39f-8476073c185f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:38.235010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1013/2029] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:38.238479 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:38.255658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b93dc4-a0dc-4c0f-9cfe-1899ac63fada admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:38.269100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:38.270771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:38.271063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:38.271343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] PATH_INFO: `/v3/projects/f4ec509d9b6f4a3392a61ce83282f71a/users/6fa1ada31f08492a8677107d35c2aae0/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:38.275238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ec509d9b6f4a3392a61ce83282f71a, user_id=6fa1ada31f08492a8677107d35c2aae0, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:38.280080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:38.313814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1017/2030] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:38 2022] PUT /identity/v3/projects/f4ec509d9b6f4a3392a61ce83282f71a/users/6fa1ada31f08492a8677107d35c2aae0/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:38.327450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:38.330273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:38.330564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:38.376905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] MFA Rules not processed for user `6fa1ada31f08492a8677107d35c2aae0`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:38.378138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:38.546453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1014/2031] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:38.559822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:38.561963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:38.669066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0384f4-11d7-4829-af9e-88eb50fb6465 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:38.678937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c377faa-63ce-4b9c-acfe-bacbae1bc821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:38.679598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c377faa-63ce-4b9c-acfe-bacbae1bc821 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:38.679873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c377faa-63ce-4b9c-acfe-bacbae1bc821 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:38.680200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c377faa-63ce-4b9c-acfe-bacbae1bc821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:38.690839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c377faa-63ce-4b9c-acfe-bacbae1bc821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:38.692451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c377faa-63ce-4b9c-acfe-bacbae1bc821 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:38.705893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1018/2032] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:38.822175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:38.839182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c316cd-9708-4818-a6fb-f7f50601c80a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:38.852108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:38.852838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:38.853083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:38.853358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:38.941348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:38.944604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:38.965494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1015/2033] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:39.815979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:39.818978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:39.819627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:39.860039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] MFA Rules not processed for user `18a2a873aeae41de8b25c2095c645479`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:39.861381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:39.961659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:39.962793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1019/2034] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:39.967050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:40.119531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a75b-fdc0-47fc-9ce6-4336c533f3df service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:40.133762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:40.135365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:40.135956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:40.136422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:40.137410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:40.140969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:40.206837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1016/2035] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:39 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:40.208920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:40.229826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e7561-5892-4b6a-99f1-3a536dfa92b2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:40.325169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:40.326664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:40.327180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:40.327657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:40.328588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:40.331593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:40.365919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1020/2036] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:40 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:40.368143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:40.383884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658889b-a444-41e2-8734-12966050f66e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:40.389791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:40.390901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:40.391272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:40.391633 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:40.392412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:40.394414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:40.406541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1017/2037] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:40.411852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:40.426676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d776762-0be8-4aaf-aa0c-40ef0c1b64d4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:40.439582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:40.441008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:40.441514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:40.442024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] PATH_INFO: `/v3/projects/aec1c5cf63d74c6c8cb11dee348744b1/users/335e69de31144d358a137db17177a038/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:40.470721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec1c5cf63d74c6c8cb11dee348744b1, user_id=335e69de31144d358a137db17177a038, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:40.474130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:40.498528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1021/2038] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:40 2022] PUT /identity/v3/projects/aec1c5cf63d74c6c8cb11dee348744b1/users/335e69de31144d358a137db17177a038/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:40.499580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:40.511569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303344-bb07-4fe4-b6f8-80a6d5749f90 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:40.593061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:40.593863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:40.594499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:40.594919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:40.596304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:40.599009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:40.629399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:40.639426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1018/2039] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:40.644805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800f8c7-16c5-470c-a35f-cbc084bcfd14 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:40.672694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95fb79-71c3-4a0a-bb74-077448588267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:40.673602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95fb79-71c3-4a0a-bb74-077448588267 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:40.673892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95fb79-71c3-4a0a-bb74-077448588267 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:40.674171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95fb79-71c3-4a0a-bb74-077448588267 admin admin] PATH_INFO: `/v3/projects/aec1c5cf63d74c6c8cb11dee348744b1/users/335e69de31144d358a137db17177a038/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:40.676342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95fb79-71c3-4a0a-bb74-077448588267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec1c5cf63d74c6c8cb11dee348744b1, user_id=335e69de31144d358a137db17177a038, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:40.680992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95fb79-71c3-4a0a-bb74-077448588267 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:40.695470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1022/2040] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:40 2022] PUT /identity/v3/projects/aec1c5cf63d74c6c8cb11dee348744b1/users/335e69de31144d358a137db17177a038/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:42.347301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:42.349704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:42.368880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469ffb-de8e-40af-af6e-171a5c734a62 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:42.508572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:42.510487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:42.510993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:42.513002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] PATH_INFO: `/v3/users/8d0e7a7b5f3a420aa1faf9d88bd8678b` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:42.516380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0e7a7b5f3a420aa1faf9d88bd8678b)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:42.519334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:42.605402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0e7a7b5f3a420aa1faf9d88bd8678b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:42.622376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0e7a7b5f3a420aa1faf9d88bd8678b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:42.625060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0e7a7b5f3a420aa1faf9d88bd8678b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:42.643692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0e7a7b5f3a420aa1faf9d88bd8678b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:42.644471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0e7a7b5f3a420aa1faf9d88bd8678b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:42.679338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1019/2041] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:42 2022] DELETE /identity/v3/users/8d0e7a7b5f3a420aa1faf9d88bd8678b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:42.684464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:42.685013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:42.685261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:42.717402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:42.718773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:42.900741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1023/2042] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:42.947447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:42.949233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:43.072331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710002f7-4fde-4fe2-8d89-23e834c814b8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:43.078708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e497cd4-502e-4fa1-a3c5-a84a443231b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:43.079384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e497cd4-502e-4fa1-a3c5-a84a443231b1 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:43.082260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e497cd4-502e-4fa1-a3c5-a84a443231b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:43.082260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e497cd4-502e-4fa1-a3c5-a84a443231b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:43.105844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e497cd4-502e-4fa1-a3c5-a84a443231b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:43.110708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e497cd4-502e-4fa1-a3c5-a84a443231b1 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:43.143635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1020/2043] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:44.201168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:44.203322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:44.221014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04f91-809b-42bc-a4ee-27d86b81cc79 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:44.237701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:44.239429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:44.242477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:44.243187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] PATH_INFO: `/v3/projects/3bd8178cdc924ded873c5f7b7fba2e48` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:44.248883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd8178cdc924ded873c5f7b7fba2e48)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:44.256012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:44.414289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd8178cdc924ded873c5f7b7fba2e48', 'request_id': 'req-71806300-eae0-462f-b07a-c149aeb6dd11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.425319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd8178cdc924ded873c5f7b7fba2e48', 'request_id': 'req-71806300-eae0-462f-b07a-c149aeb6dd11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.437800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd8178cdc924ded873c5f7b7fba2e48', 'request_id': 'req-71806300-eae0-462f-b07a-c149aeb6dd11'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.463314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] The token cache is being invalidate because project 3bd8178cdc924ded873c5f7b7fba2e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:44.463962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.466081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-71806300-eae0-462f-b07a-c149aeb6dd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.471230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1024/2044] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:44 2022] DELETE /identity/v3/projects/3bd8178cdc924ded873c5f7b7fba2e48 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:44.476326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:44.476864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:44.477319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:44.511774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:44.516076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:44.724063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1021/2045] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:44.727097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:44.727457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:44.727809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:44.747084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:44.755731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:44.766552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:44.768261 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:44.779202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205e031-cb95-408a-a57e-d51ba5d184d9 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:44.793514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1025/2046] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:44.797233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:44.799907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:44.800850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:44.801132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:44.801447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] PATH_INFO: `/v3/users/18e2bfa909334e87bb742767a4e2bd77` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:44.802719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e2bfa909334e87bb742767a4e2bd77)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:44.804729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:44.813914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ccfa6-80b2-45d5-a94c-b8a8c4f252f4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:44.819764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:44.820518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:44.821518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:44.823012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:44.823918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:44.824377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:44.828243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:44.846348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1026/2047] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:52:44.853258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:44.879410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7962-1094-4cd1-9c08-5f67be506660 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:44.885614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:44.887662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:44.887662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:44.887662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:44.887662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:44.912811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:44.920038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e2bfa909334e87bb742767a4e2bd77'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.929381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e2bfa909334e87bb742767a4e2bd77'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.930313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e2bfa909334e87bb742767a4e2bd77'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.943290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e2bfa909334e87bb742767a4e2bd77'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.943953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b952b835-f54e-41ce-a1ac-d3e5ade7d22a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e2bfa909334e87bb742767a4e2bd77'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:44.972889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1027/2048] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:44 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:44.976383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:44.978306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:45.006679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1022/2049] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:44 2022] DELETE /identity/v3/users/18e2bfa909334e87bb742767a4e2bd77 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:45.015438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.018413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.019412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.070854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:45.072839 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:45.112516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c3a54-2ce5-4fba-b3b7-658ec4a4b39a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:45.233478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1023/2050] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:45.247681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:45.249464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:45.251120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.251338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:45.251934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.252908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.254868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:45.257073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:45.287942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1028/2051] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:44 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:45.289120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:45.329639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff42b9-8b13-4765-8275-e4eee62c8c0f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:45.338209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:45.339139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.339432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.339712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.340503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:45.342222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:45.358481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1029/2052] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:45.359917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:45.365807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef0baa-25ee-4df5-8454-5802353e58f5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:45.371665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ce921-7082-4121-84b9-4d0ce397e288 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:45.374389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230133d-3a1c-433d-ad1a-d847a13ed4d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:45.375254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230133d-3a1c-433d-ad1a-d847a13ed4d9 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.375738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230133d-3a1c-433d-ad1a-d847a13ed4d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.376114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230133d-3a1c-433d-ad1a-d847a13ed4d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.377262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:45.378074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.378495 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.378910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.379871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:45.381674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:45.399568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1030/2053] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:45.400791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:45.409447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230133d-3a1c-433d-ad1a-d847a13ed4d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:45.411667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230133d-3a1c-433d-ad1a-d847a13ed4d9 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:45.413895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad770b64-6064-4efb-b1a6-1aeaa2d88edc admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:45.419049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:45.420137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.420557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.421028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] PATH_INFO: `/v3/projects/b3f392efe69d4c10a93c87c47f945a4c/users/7b508c172eb649068979af03fe0ebe85/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.438544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1024/2054] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:45.453591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f392efe69d4c10a93c87c47f945a4c, user_id=7b508c172eb649068979af03fe0ebe85, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:45.461124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:45.501308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.501820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.501968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.521161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1031/2055] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:45 2022] PUT /identity/v3/projects/b3f392efe69d4c10a93c87c47f945a4c/users/7b508c172eb649068979af03fe0ebe85/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:45.571858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] MFA Rules not processed for user `7b508c172eb649068979af03fe0ebe85`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:45.573255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:45.761624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1025/2056] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:45.773954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:45.777201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:45.912152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a699d-f4a4-495d-9efd-4d00753a482c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:45.916685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:45.917538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:45.917970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:45.918711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:45.941138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:45.943951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:45.957070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1032/2057] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:46.171444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:46.173381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:46.375271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ad643-19db-468d-a1ec-e5b8f06bb42b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:46.385372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:46.388811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:46.389305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:46.389757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] PATH_INFO: `/v3/projects/b734bb722ffd4dd49fb449936221e5e5` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:46.393997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b734bb722ffd4dd49fb449936221e5e5)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:46.409766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:46.573072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b734bb722ffd4dd49fb449936221e5e5', 'request_id': 'req-82294fa2-f859-43f5-a0b7-ae98c5ec4697'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.581792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b734bb722ffd4dd49fb449936221e5e5', 'request_id': 'req-82294fa2-f859-43f5-a0b7-ae98c5ec4697'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.606998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b734bb722ffd4dd49fb449936221e5e5', 'request_id': 'req-82294fa2-f859-43f5-a0b7-ae98c5ec4697'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.650399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] The token cache is being invalidate because project b734bb722ffd4dd49fb449936221e5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:46.650703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.655442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.661498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1026/2058] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:46 2022] DELETE /identity/v3/projects/b734bb722ffd4dd49fb449936221e5e5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:46.662542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:46.682390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7aa36-78ca-4fb8-9cb0-7bb66a83341a service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:46.842540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:46.843536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:46.844018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:46.844342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] PATH_INFO: `/v3/users/6fa1ada31f08492a8677107d35c2aae0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:46.846551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa1ada31f08492a8677107d35c2aae0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:46.852171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:46.973879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa1ada31f08492a8677107d35c2aae0'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.974617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa1ada31f08492a8677107d35c2aae0'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.982136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa1ada31f08492a8677107d35c2aae0'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.992607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa1ada31f08492a8677107d35c2aae0'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:46.993431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-47e6ce50-5d2d-4af3-a816-894bae0a8992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa1ada31f08492a8677107d35c2aae0'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:47.016136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1033/2059] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:46 2022] DELETE /identity/v3/users/6fa1ada31f08492a8677107d35c2aae0 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:47.627003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:47.629085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:47.656533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82294fa2-f859-43f5-a0b7-ae98c5ec4697 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:47.772767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:47.774520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:47.774896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:47.775296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] PATH_INFO: `/v3/projects/f4ec509d9b6f4a3392a61ce83282f71a` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:47.777822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ec509d9b6f4a3392a61ce83282f71a)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:47.780433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:47.869198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ec509d9b6f4a3392a61ce83282f71a', 'request_id': 'req-c797d34f-3a65-4d33-bbea-aed522580288'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:47.880172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ec509d9b6f4a3392a61ce83282f71a', 'request_id': 'req-c797d34f-3a65-4d33-bbea-aed522580288'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:47.896692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ec509d9b6f4a3392a61ce83282f71a', 'request_id': 'req-c797d34f-3a65-4d33-bbea-aed522580288'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:47.930643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] The token cache is being invalidate because project f4ec509d9b6f4a3392a61ce83282f71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:47.930967 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:47.931883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c797d34f-3a65-4d33-bbea-aed522580288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:47.937092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1027/2060] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:47 2022] DELETE /identity/v3/projects/f4ec509d9b6f4a3392a61ce83282f71a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:47.942115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:47.943110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:47.943568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:47.975841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:47.980577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:48.112116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1034/2061] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:48.121126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.121427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.121759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.169757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:48.171277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:48.190518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1028/2062] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:48.195532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:48.208542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692210fa-2538-4807-abd0-db1c034c97f9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:48.223544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:48.225033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.225326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.225605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.228149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:48.228468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:48.236503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:48.277824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:48.301897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1035/2063] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:52:48.304751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2355c-1396-48b4-9c1b-43159c5e8594 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:48.314459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:48.315338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.316450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.317429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.329591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:48.332256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:48.395194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:48.402595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1029/2064] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:48 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:48.408113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaae7e-82a5-4c44-892e-767537cc6cc6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:48.557543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:48.558492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.559557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.562045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.563128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:48.567354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:48.633679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1036/2065] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:48 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:48.638537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:48.659467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd7895-0cf7-4861-873d-cbd18de82c4d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:48.674262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:48.678285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.678963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.679396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.680475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:48.686565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:48.724163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1030/2066] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:48.724682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:48.739774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3274-1c5b-4354-ae33-e1710d0ab791 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:48.752335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828173f-ada3-4332-a82f-6e7a65d022cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:48.753220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828173f-ada3-4332-a82f-6e7a65d022cb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.753784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828173f-ada3-4332-a82f-6e7a65d022cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.754840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828173f-ada3-4332-a82f-6e7a65d022cb admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.755875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828173f-ada3-4332-a82f-6e7a65d022cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:48.757816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828173f-ada3-4332-a82f-6e7a65d022cb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:48.772464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1037/2067] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:48.776501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:48.792820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1e464-7c69-4f6c-b697-a5c5b19efe4d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:48.803107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:48.805902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.806195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.806477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] PATH_INFO: `/v3/projects/7985d34aadcf4f08837da4d2039dd9e8/users/06121914510844929bcaaaf85f8d55c1/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.825868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7985d34aadcf4f08837da4d2039dd9e8, user_id=06121914510844929bcaaaf85f8d55c1, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:48.830477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:48.852238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1031/2068] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:48 2022] PUT /identity/v3/projects/7985d34aadcf4f08837da4d2039dd9e8/users/06121914510844929bcaaaf85f8d55c1/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:48.862772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:48.864311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:48.864844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:48.899050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] MFA Rules not processed for user `06121914510844929bcaaaf85f8d55c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:48.900588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:48.990326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1038/2069] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:48.994984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.101842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962f183-b475-4c09-b62a-1deda51ee10e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.121634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.122318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.122598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.122868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.123563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.125525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.202568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1032/2070] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:48 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:49.205150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.228146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d3e6-6360-49b9-91fa-e26c9e753729 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.345340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.346038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.346315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.346761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.347618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.350292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.389215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1039/2071] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:49 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:49.393387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.408624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23f11-b135-4409-b17e-0ce7192e78de admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.429225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.430040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.430425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.430783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.431661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.436755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.460463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1033/2072] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:49.465033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.483036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168ca7-9077-4dd1-b1ae-b4c3d868a91d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.510205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023b2e9-6596-452f-9d46-69ad68c3c165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.511166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023b2e9-6596-452f-9d46-69ad68c3c165 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.511624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023b2e9-6596-452f-9d46-69ad68c3c165 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.512049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023b2e9-6596-452f-9d46-69ad68c3c165 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.513009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023b2e9-6596-452f-9d46-69ad68c3c165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.515186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023b2e9-6596-452f-9d46-69ad68c3c165 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.532937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1040/2073] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:49.533854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.551747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1abea0-e477-4713-9361-83c7835a81d0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.564768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.566332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.567482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.567768 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] PATH_INFO: `/v3/projects/dccaff0cb7e94d2f933ac5130f9c356b/users/b1aa121d68ea4603a9aa3dde25ef3089/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.592936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccaff0cb7e94d2f933ac5130f9c356b, user_id=b1aa121d68ea4603a9aa3dde25ef3089, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.595969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.634801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1034/2074] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:49 2022] PUT /identity/v3/projects/dccaff0cb7e94d2f933ac5130f9c356b/users/b1aa121d68ea4603a9aa3dde25ef3089/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:49.638077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.638597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.639024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.671517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] MFA Rules not processed for user `b1aa121d68ea4603a9aa3dde25ef3089`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:49.672782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:49.737257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.741242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:49.794108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1041/2075] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:49.795098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:49.845912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bece5-aaa5-454b-8e20-9d3956c6c30a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.854642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.855782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.857507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.857846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] PATH_INFO: `/v3/users/53c05b0f094644888f7444f917503a88` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.859870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c05b0f094644888f7444f917503a88)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.864832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.933068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7bcbb-cd4d-4457-8411-9563e15107e1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:49.948972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:49.949794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:49.950075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:49.950449 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:49.951692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:49.956275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:49.960647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c05b0f094644888f7444f917503a88'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:49.978172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c05b0f094644888f7444f917503a88'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:49.979468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c05b0f094644888f7444f917503a88'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:49.992069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c05b0f094644888f7444f917503a88'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:49.993390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ffe4e802-0035-4003-bf29-e18d05d8cb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c05b0f094644888f7444f917503a88'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:50.012602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1042/2076] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:49 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:50.016158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.020733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1035/2077] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:49 2022] DELETE /identity/v3/users/53c05b0f094644888f7444f917503a88 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:50.020891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.141913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f53222-e052-4cdd-becb-34cf49bcc778 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.220356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.221482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.224541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.271471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.272168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.272448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.272773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.273470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.275686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.277827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] MFA Rules not processed for user `335e69de31144d358a137db17177a038`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:50.279223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.325913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1043/2078] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/users => generated 295 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:50.329769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.351806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb80199-91b6-4cf2-9530-de9149c05eb2 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.366920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.374708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.375091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.375405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.382120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.383071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.416871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1044/2079] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:50.416871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.425131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.434653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1036/2080] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:50.437486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.447008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76598e28-77da-4d16-8a61-a7be6f20efcb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.453121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910421b-c1e1-4d19-a2a4-a95e5e01a600 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.454962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.455888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.456173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.456505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] PATH_INFO: `/v3/projects/3d4d78fd087744ecbc5e55e2c4050974` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.458222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4d78fd087744ecbc5e55e2c4050974)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.460153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.460757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.461577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.462054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.462519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] PATH_INFO: `/v3/projects/5b8c1d7ac64f463b87f737a26f1a1f6e/users/b6eaaf036bdd4566bc48374ae481d7af/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.499293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8c1d7ac64f463b87f737a26f1a1f6e, user_id=b6eaaf036bdd4566bc48374ae481d7af, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.502441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.530979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1037/2081] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:50 2022] PUT /identity/v3/projects/5b8c1d7ac64f463b87f737a26f1a1f6e/users/b6eaaf036bdd4566bc48374ae481d7af/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:50.532480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.537414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.552597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4d78fd087744ecbc5e55e2c4050974', 'request_id': 'req-4f58ba38-b723-4403-8911-20d99dfb4a8c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:50.564718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4d78fd087744ecbc5e55e2c4050974', 'request_id': 'req-4f58ba38-b723-4403-8911-20d99dfb4a8c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:50.591146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4d78fd087744ecbc5e55e2c4050974', 'request_id': 'req-4f58ba38-b723-4403-8911-20d99dfb4a8c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:50.614911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] The token cache is being invalidate because project 3d4d78fd087744ecbc5e55e2c4050974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:50.618986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:50.621623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:50.623278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5327-b343-4241-9ccb-0439b1c6d78f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.626592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1045/2082] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:50 2022] DELETE /identity/v3/projects/3d4d78fd087744ecbc5e55e2c4050974 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:50.626592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.626592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.698372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.699021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.699309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.699583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.701937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.716930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58ba38-b723-4403-8911-20d99dfb4a8c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.726769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd570e-d722-477b-b1ba-19312da52ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.727370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd570e-d722-477b-b1ba-19312da52ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.727656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd570e-d722-477b-b1ba-19312da52ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.727929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd570e-d722-477b-b1ba-19312da52ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.728682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd570e-d722-477b-b1ba-19312da52ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.730281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd570e-d722-477b-b1ba-19312da52ba4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.740553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1046/2083] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:50.742104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.742396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.742674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.760539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:50.761777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.766678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.768113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69c2e9-5899-4959-8a01-f3dbe686e8d9 service glance] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.770386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1047/2084] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:50.770962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.777841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1038/2085] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:50.779205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.779494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.779769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.782900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483576b-4007-47e0-8fff-fd8fa44120c5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.788277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325703-e717-4b2c-8e4a-90bac66f14ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.789253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325703-e717-4b2c-8e4a-90bac66f14ec admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.789531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325703-e717-4b2c-8e4a-90bac66f14ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.789802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325703-e717-4b2c-8e4a-90bac66f14ec admin admin] PATH_INFO: `/v3/projects/5b8c1d7ac64f463b87f737a26f1a1f6e/users/b6eaaf036bdd4566bc48374ae481d7af/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.791758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04325703-e717-4b2c-8e4a-90bac66f14ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8c1d7ac64f463b87f737a26f1a1f6e, user_id=b6eaaf036bdd4566bc48374ae481d7af, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.794463 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04325703-e717-4b2c-8e4a-90bac66f14ec admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.797949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:50.799156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.807334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1039/2086] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:50.811400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.814034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1048/2087] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:50 2022] PUT /identity/v3/projects/5b8c1d7ac64f463b87f737a26f1a1f6e/users/b6eaaf036bdd4566bc48374ae481d7af/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:50.820726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f8994-617f-4b17-bbee-4764bcd99409 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.828558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.828875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.829160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.879390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] MFA Rules not processed for user `b6eaaf036bdd4566bc48374ae481d7af`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:50.879390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:50.898597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.899320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.899558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.899817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.900577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:50.900907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.902495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.916890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1040/2088] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:52:50.920805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.931265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6331522-a7d3-4654-a070-2ea37d13d112 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.935986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba731e-cf69-460d-97e0-e655157f7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:50.936670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba731e-cf69-460d-97e0-e655157f7dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.936939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba731e-cf69-460d-97e0-e655157f7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.937214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba731e-cf69-460d-97e0-e655157f7dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.938108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba731e-cf69-460d-97e0-e655157f7dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:50.939868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba731e-cf69-460d-97e0-e655157f7dc0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:50.974425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1041/2089] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:50.974425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:50.974425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:50.974425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:50.977517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1049/2090] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:50.978197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:50.988214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d52f20-69b1-4b82-883d-5b8daaab300d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:50.991805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:50.993014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:51.043549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.044204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.044487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.044842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.045519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.046042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1042/2091] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:51.046649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:51.047462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.047912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:51.089539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1050/2092] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:50 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:51.095011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:51.104863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e85ec-11a8-402c-afa9-08f01946102f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:51.111653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.112311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.112588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.112892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.113610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.115170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.115727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28c70e-cc5d-4396-997a-8549a8a14e85 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:51.120478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.121182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.121430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.121707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.123674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:51.128811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1051/2093] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:51.128811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:51.130696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:51.198244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.199680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.208198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7c95e-79ec-4f44-82a9-e2362fbca02a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:51.209587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1043/2094] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:51.210250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:51.220188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e407df-36db-46f9-bf74-51a1fa55a41b service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:51.224080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e4fc0-2a70-4649-b03d-c90889dcb6f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.224908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.225228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e4fc0-2a70-4649-b03d-c90889dcb6f8 service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.225561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.225809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e4fc0-2a70-4649-b03d-c90889dcb6f8 service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.225916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.226166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.226448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e4fc0-2a70-4649-b03d-c90889dcb6f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.226892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.228468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.237371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e4fc0-2a70-4649-b03d-c90889dcb6f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.241505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e4fc0-2a70-4649-b03d-c90889dcb6f8 service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.241646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1044/2095] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:51.245398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:51.251052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1052/2096] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:52:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:52:51.258541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d40ff8-4481-4d0b-8d9a-ba68f774a688 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:51.263954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.265082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.265436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.265776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] PATH_INFO: `/v3/projects/9acbb1114a8b4d0fb9c989ff709adf3c/users/05adc21555034c94ba59e90a0b20010c/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.284266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acbb1114a8b4d0fb9c989ff709adf3c, user_id=05adc21555034c94ba59e90a0b20010c, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.288407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.312272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1045/2097] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:51 2022] PUT /identity/v3/projects/9acbb1114a8b4d0fb9c989ff709adf3c/users/05adc21555034c94ba59e90a0b20010c/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:51.325810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.326155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.326384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.385773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] MFA Rules not processed for user `05adc21555034c94ba59e90a0b20010c`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:51.389361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:51.501056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1053/2098] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:51.516924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:51.519306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:51.702168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48856-bfec-484b-933b-b70b597b01b0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:51.733299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:51.734044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:51.734431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:51.734806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:51.746250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:51.748241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:51.764298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1046/2099] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:53.147910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:53.260342 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd4c7-e66e-4432-9629-d784f0cda3b0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:53.276359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:53.277316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:53.277701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:53.280677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:53.282787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:53.422988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:53.424468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:53.440142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1054/2100] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:54.499889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:54.511736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87388-a73a-44db-93a2-f4cea57192c8 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:54.519040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fd6c7-0fbe-4d2f-a644-38e8b4e3b20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:54.520225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fd6c7-0fbe-4d2f-a644-38e8b4e3b20d service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:54.520764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fd6c7-0fbe-4d2f-a644-38e8b4e3b20d service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:54.521262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fd6c7-0fbe-4d2f-a644-38e8b4e3b20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:54.605164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fd6c7-0fbe-4d2f-a644-38e8b4e3b20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:54.606936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72fd6c7-0fbe-4d2f-a644-38e8b4e3b20d service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:54.620412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1047/2101] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:55.073861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:55.190747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927a4d7-e1cb-41c0-8710-4a14fd652dc6 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:55.195967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:55.197057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:55.197444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:55.197841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] PATH_INFO: `/v3/users/05adc21555034c94ba59e90a0b20010c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:55.200157 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05adc21555034c94ba59e90a0b20010c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:55.202304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:55.288755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05adc21555034c94ba59e90a0b20010c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:55.289566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05adc21555034c94ba59e90a0b20010c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:55.297651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05adc21555034c94ba59e90a0b20010c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:55.314386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05adc21555034c94ba59e90a0b20010c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:55.315211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05adc21555034c94ba59e90a0b20010c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:55.340677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1055/2102] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:55 2022] DELETE /identity/v3/users/05adc21555034c94ba59e90a0b20010c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:55.343431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:55.343720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:55.343999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:55.370296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:55.371874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:55.467570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1048/2103] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:55.478926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:55.480755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:55.580332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e49c9e-9edb-48ac-b0b2-1aa5db681a14 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:55.589739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda66ad-353a-409c-ab8b-144b5c034354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:55.590645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda66ad-353a-409c-ab8b-144b5c034354 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:55.591032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda66ad-353a-409c-ab8b-144b5c034354 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:55.591430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda66ad-353a-409c-ab8b-144b5c034354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:55.602932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda66ad-353a-409c-ab8b-144b5c034354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:55.604581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda66ad-353a-409c-ab8b-144b5c034354 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:55.657246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1056/2104] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:52:56.151745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.154521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:56.175160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffcc19-76b3-40ad-b149-744a3297d52c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.187949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.188878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.189127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.189388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] PATH_INFO: `/v3/projects/9acbb1114a8b4d0fb9c989ff709adf3c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.203153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acbb1114a8b4d0fb9c989ff709adf3c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.210423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.287148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acbb1114a8b4d0fb9c989ff709adf3c', 'request_id': 'req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:56.292994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acbb1114a8b4d0fb9c989ff709adf3c', 'request_id': 'req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:56.302389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acbb1114a8b4d0fb9c989ff709adf3c', 'request_id': 'req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:56.331378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] The token cache is being invalidate because project 9acbb1114a8b4d0fb9c989ff709adf3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:56.331879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:56.333574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-f99815b8-ba08-4d3f-8289-be7d7ec06bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:56.336846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1049/2105] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:56 2022] DELETE /identity/v3/projects/9acbb1114a8b4d0fb9c989ff709adf3c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:56.341647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.342653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.343172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.391176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:56.395555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:56.501987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1057/2106] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:56.510543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.510967 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.511368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.534713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:56.537069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:56.551411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.552243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1050/2107] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:56.569593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc223e9-d935-4388-a532-ab4acbc54987 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.590284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.591068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.591450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.591824 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.592766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:52:56.593166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.602978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.621995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.628038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1058/2108] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:52:56.632052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0c5a2-c838-472d-a7e8-06b646757883 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.637381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.638317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.638697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.639085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.639921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.642288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.685230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.688845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1051/2109] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:56 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:56.703250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ea9b9-1f23-45d5-99b4-cf407b957a48 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.771062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.771729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.772008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.772276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.773061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.775141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.804432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1059/2110] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:56 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:56.805976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.819602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76202f-b667-47c7-aa6f-4b8db80570c6 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.825106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.826017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.826460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.827006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.827891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.829571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.842280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1052/2111] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:56.846317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.859552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb0d7-8bd8-4b3c-9696-cd67684adfe8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.868300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568422-52eb-44d5-af88-96e3c9024fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.869416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568422-52eb-44d5-af88-96e3c9024fee admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.870124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568422-52eb-44d5-af88-96e3c9024fee admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.870619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568422-52eb-44d5-af88-96e3c9024fee admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.871591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3568422-52eb-44d5-af88-96e3c9024fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.873978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3568422-52eb-44d5-af88-96e3c9024fee admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.887147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1060/2112] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:52:56.891269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:56.903067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b38067-0c12-4036-9d39-5215d43b4f3c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:56.919062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:56.920328 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.920873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.921649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] PATH_INFO: `/v3/projects/d5b974f6fd6e4d1e974a9b42d93bcff5/users/cea4bc97a05c4ffe937871e162f0db2c/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:56.940388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b974f6fd6e4d1e974a9b42d93bcff5, user_id=cea4bc97a05c4ffe937871e162f0db2c, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:56.944070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:56.969246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1053/2113] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:56 2022] PUT /identity/v3/projects/d5b974f6fd6e4d1e974a9b42d93bcff5/users/cea4bc97a05c4ffe937871e162f0db2c/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:56.981402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:56.981749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:56.982049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.016139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] MFA Rules not processed for user `cea4bc97a05c4ffe937871e162f0db2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:57.017435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:57.139949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1061/2114] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:57.145092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.303493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519428-9aa4-4488-9b86-66852bb7e6df admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:57.309721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:57.310809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.311324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.311805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.312740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:57.315193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:57.379012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1054/2115] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:52:57 2022] POST /identity/v3/projects => generated 405 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:52:57.382718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.398514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcfdd-149a-4231-99c5-d8868a16da1c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:57.521661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:57.522365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.522613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.523059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.523893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:57.526630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:57.580453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.580863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1062/2116] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:52:57 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:52:57.609664 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfb51b-27a5-43e6-9096-c78794aa0c71 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:57.615609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:57.616449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.616948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.617426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.618341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:57.620570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:57.635314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1055/2117] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:57.639304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.656968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7578d-be73-482d-8679-28bcfc7ec563 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:57.663981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:57.665370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.665666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.665981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] PATH_INFO: `/v3/projects/ef7b5b78a4014c629e499d4ddd29a3f8/users/126e8e9f571144dd9650e90230d5db0b/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.686334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7b5b78a4014c629e499d4ddd29a3f8, user_id=126e8e9f571144dd9650e90230d5db0b, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:57.689119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:57.706303 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1063/2118] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:57 2022] PUT /identity/v3/projects/ef7b5b78a4014c629e499d4ddd29a3f8/users/126e8e9f571144dd9650e90230d5db0b/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:57.710670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.720515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0181b-67c2-48e9-a356-3f846220f771 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:57.787454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2595-734f-4ece-8996-89ce7e55bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:57.788124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2595-734f-4ece-8996-89ce7e55bd3c admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.788411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2595-734f-4ece-8996-89ce7e55bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.788724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2595-734f-4ece-8996-89ce7e55bd3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.789445 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2595-734f-4ece-8996-89ce7e55bd3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:57.791041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2595-734f-4ece-8996-89ce7e55bd3c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:57.804678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1056/2119] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:52:57.807805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.819633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0bae7-08cd-45bd-8fcd-6f1cb2ac473d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:57.825004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:57.826119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.826473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.826803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] PATH_INFO: `/v3/projects/ef7b5b78a4014c629e499d4ddd29a3f8/users/126e8e9f571144dd9650e90230d5db0b/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.828907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7b5b78a4014c629e499d4ddd29a3f8, user_id=126e8e9f571144dd9650e90230d5db0b, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:57.831754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:57.846444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1064/2120] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:52:57 2022] PUT /identity/v3/projects/ef7b5b78a4014c629e499d4ddd29a3f8/users/126e8e9f571144dd9650e90230d5db0b/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:57.858532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:57.858830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:57.859113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:57.886770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] MFA Rules not processed for user `126e8e9f571144dd9650e90230d5db0b`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:57.887966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:57.977275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1057/2121] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:57.989245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:57.990921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:58.063514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1863913-632f-4ff4-ba65-0a4d9fe1ed47 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:58.071092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b205888-b235-4341-a735-2b6398b9836e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:58.071894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b205888-b235-4341-a735-2b6398b9836e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.072589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b205888-b235-4341-a735-2b6398b9836e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.072978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b205888-b235-4341-a735-2b6398b9836e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.084351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b205888-b235-4341-a735-2b6398b9836e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:58.085908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b205888-b235-4341-a735-2b6398b9836e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:58.109993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1065/2122] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:57 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:58.283832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:58.366818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21a7da-43d5-483d-80b4-869039a50d9a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:58.373899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:58.375230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.375583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.375926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] PATH_INFO: `/v3/users/cea4bc97a05c4ffe937871e162f0db2c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.377155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea4bc97a05c4ffe937871e162f0db2c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:58.379142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:58.434579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea4bc97a05c4ffe937871e162f0db2c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.441863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea4bc97a05c4ffe937871e162f0db2c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.442611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea4bc97a05c4ffe937871e162f0db2c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.453477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea4bc97a05c4ffe937871e162f0db2c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.454291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea4bc97a05c4ffe937871e162f0db2c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.476092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1058/2123] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:58 2022] DELETE /identity/v3/users/cea4bc97a05c4ffe937871e162f0db2c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:58.480880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.481388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.481492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.501608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:58.503051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:58.575189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1066/2124] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:58.585803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:58.587457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:58.638426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:58.639963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:58.653467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47261ae9-64ab-49c4-8ef3-618569f89216 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:58.658219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:58.659580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.660136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.660716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] PATH_INFO: `/v3/users/7b508c172eb649068979af03fe0ebe85` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.662222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b508c172eb649068979af03fe0ebe85)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:58.664396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:58.677718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8279510-c3d4-4908-9cb8-26bc2f21327e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:58.683665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:58.684327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.684924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.685205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.700742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:58.702192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:58.713741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1059/2125] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:58.759067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b508c172eb649068979af03fe0ebe85'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.759583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b508c172eb649068979af03fe0ebe85'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.767607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b508c172eb649068979af03fe0ebe85'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.781060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b508c172eb649068979af03fe0ebe85'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.782168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-924a135f-0e7d-4ff8-96ef-ec38448af966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b508c172eb649068979af03fe0ebe85'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:58.791573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:58.797330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:58.816766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1067/2126] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:58 2022] DELETE /identity/v3/users/7b508c172eb649068979af03fe0ebe85 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:58.819242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574916e-f192-4f9f-b716-a63efb46c843 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:58.820406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.820943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.821507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.850535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:58.852140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:58.951644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:58.952568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:58.952822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:58.955253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] PATH_INFO: `/v3/users/06121914510844929bcaaaf85f8d55c1` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:58.964063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=06121914510844929bcaaaf85f8d55c1)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:58.966055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:58.969847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1068/2127] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:52:58.982070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:58.983538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.092575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09befa03-db4d-4dbb-8b54-c794c1b16879 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:59.100361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06121914510844929bcaaaf85f8d55c1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.104145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:59.104145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:59.104408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:59.104683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:59.115509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06121914510844929bcaaaf85f8d55c1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.116370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06121914510844929bcaaaf85f8d55c1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.125248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:59.126696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:59.138434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06121914510844929bcaaaf85f8d55c1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.139593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c1109b7a-ed83-4546-bf0e-93e244dddcfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06121914510844929bcaaaf85f8d55c1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.152480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1069/2128] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:52:59.153493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:59.157442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.166895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1060/2129] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:58 2022] DELETE /identity/v3/users/06121914510844929bcaaaf85f8d55c1 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:52:59.176236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5edb4b-613d-4862-829f-18de571cbccc service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:59.176700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:59.177247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:59.177750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:59.216413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:52:59.224075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.284856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:59.286139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:59.286412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:59.286684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] PATH_INFO: `/v3/projects/d5b974f6fd6e4d1e974a9b42d93bcff5` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:59.293277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b974f6fd6e4d1e974a9b42d93bcff5)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:59.295206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:59.342427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1061/2130] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:52:59.358256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:59.359671 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.398321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b974f6fd6e4d1e974a9b42d93bcff5', 'request_id': 'req-83a62abc-2931-4ee2-bbf1-ca795c8862b2'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.404088 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b974f6fd6e4d1e974a9b42d93bcff5', 'request_id': 'req-83a62abc-2931-4ee2-bbf1-ca795c8862b2'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.417248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b974f6fd6e4d1e974a9b42d93bcff5', 'request_id': 'req-83a62abc-2931-4ee2-bbf1-ca795c8862b2'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.431495 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] The token cache is being invalidate because project d5b974f6fd6e4d1e974a9b42d93bcff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:59.431920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.432813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.434549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1070/2131] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:59 2022] DELETE /identity/v3/projects/d5b974f6fd6e4d1e974a9b42d93bcff5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:59.451167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:59.457038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.562246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ecdbc-178a-4878-ba4b-95e13c962a93 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:59.606224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a62abc-2931-4ee2-bbf1-ca795c8862b2 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:59.612607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:59.614198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:59.614481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:59.614758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] PATH_INFO: `/v3/users/126e8e9f571144dd9650e90230d5db0b` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:59.616046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126e8e9f571144dd9650e90230d5db0b)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:59.618003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:59.646507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:59.647345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:59.647630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:59.647904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:59.649987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.681114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:59.682918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:59.699041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1062/2132] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:52:59.700117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:52:59.701639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:52:59.713957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126e8e9f571144dd9650e90230d5db0b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.715039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126e8e9f571144dd9650e90230d5db0b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.716802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094cb3-6472-4605-84a9-3cbd17f6db32 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:52:59.725610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126e8e9f571144dd9650e90230d5db0b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.747195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126e8e9f571144dd9650e90230d5db0b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.748027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126e8e9f571144dd9650e90230d5db0b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.749191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:52:59.752727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:52:59.753933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:52:59.756575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] PATH_INFO: `/v3/projects/b3f392efe69d4c10a93c87c47f945a4c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:52:59.765376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f392efe69d4c10a93c87c47f945a4c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:52:59.772442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:52:59.777304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1071/2133] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:52:59 2022] DELETE /identity/v3/users/126e8e9f571144dd9650e90230d5db0b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:52:59.889507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f392efe69d4c10a93c87c47f945a4c', 'request_id': 'req-dde39a36-8f1a-43ea-ab2b-e059931def2f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.895887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f392efe69d4c10a93c87c47f945a4c', 'request_id': 'req-dde39a36-8f1a-43ea-ab2b-e059931def2f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.903550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f392efe69d4c10a93c87c47f945a4c', 'request_id': 'req-dde39a36-8f1a-43ea-ab2b-e059931def2f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.929842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] The token cache is being invalidate because project b3f392efe69d4c10a93c87c47f945a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:52:59.930262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.931071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:52:59.933363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1063/2134] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:52:59 2022] DELETE /identity/v3/projects/b3f392efe69d4c10a93c87c47f945a4c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:00.218917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:00.220373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:00.270596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:00.272295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:00.323714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f29bc-a190-4533-beae-c01632e05b54 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:00.370171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde39a36-8f1a-43ea-ab2b-e059931def2f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:00.375181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:00.376479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.377096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.377240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] PATH_INFO: `/v3/projects/7985d34aadcf4f08837da4d2039dd9e8` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.379424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7985d34aadcf4f08837da4d2039dd9e8)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:00.383056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:00.385439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:00.386445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.386729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.387001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] PATH_INFO: `/v3/projects/ef7b5b78a4014c629e499d4ddd29a3f8` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.388200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7b5b78a4014c629e499d4ddd29a3f8)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:00.390159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:00.442573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7985d34aadcf4f08837da4d2039dd9e8', 'request_id': 'req-1e36649b-f710-401c-89ad-2bae02537d73'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.444069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7b5b78a4014c629e499d4ddd29a3f8', 'request_id': 'req-1af921d4-f144-4e76-a527-d34bea9462e9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.450302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7985d34aadcf4f08837da4d2039dd9e8', 'request_id': 'req-1e36649b-f710-401c-89ad-2bae02537d73'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.452302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7b5b78a4014c629e499d4ddd29a3f8', 'request_id': 'req-1af921d4-f144-4e76-a527-d34bea9462e9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.456317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7985d34aadcf4f08837da4d2039dd9e8', 'request_id': 'req-1e36649b-f710-401c-89ad-2bae02537d73'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.469988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7b5b78a4014c629e499d4ddd29a3f8', 'request_id': 'req-1af921d4-f144-4e76-a527-d34bea9462e9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.474324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] The token cache is being invalidate because project 7985d34aadcf4f08837da4d2039dd9e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:00.474788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.475631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.477910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1064/2135] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:00 2022] DELETE /identity/v3/projects/7985d34aadcf4f08837da4d2039dd9e8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:00.481435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:00.483083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:00.487051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] The token cache is being invalidate because project ef7b5b78a4014c629e499d4ddd29a3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:00.487051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.487579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1af921d4-f144-4e76-a527-d34bea9462e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.490135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1072/2136] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:00 2022] DELETE /identity/v3/projects/ef7b5b78a4014c629e499d4ddd29a3f8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:00.494665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7febf-acb2-42af-9b23-7a80526a1087 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.495069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7febf-acb2-42af-9b23-7a80526a1087 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.495413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7febf-acb2-42af-9b23-7a80526a1087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.519940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7ac7febf-acb2-42af-9b23-7a80526a1087 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:00.521286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7ac7febf-acb2-42af-9b23-7a80526a1087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:00.572878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36649b-f710-401c-89ad-2bae02537d73 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:00.602369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1073/2137] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:00.605289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.605289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.605289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.624750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:00.625923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:00.647543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1074/2138] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:53:00.648743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:00.648988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:00.649876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.650159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.650419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] PATH_INFO: `/v3/users/b1aa121d68ea4603a9aa3dde25ef3089` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.651618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1aa121d68ea4603a9aa3dde25ef3089)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:00.653574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:00.659156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac26c5-bcce-4456-938c-604e0728e9ea None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:00.664356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:00.665045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.665325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.665594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.666320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:53:00.666649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:00.668380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:00.689761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1075/2139] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:53:00.694856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:00.705203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213504d2-df31-4c40-b274-1140863d96d0 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:00.708893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1aa121d68ea4603a9aa3dde25ef3089'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.710286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:00.710940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.711216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.711486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.712172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:00.714263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:00.716384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1aa121d68ea4603a9aa3dde25ef3089'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.718993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1aa121d68ea4603a9aa3dde25ef3089'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.731938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1aa121d68ea4603a9aa3dde25ef3089'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.732675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1aa121d68ea4603a9aa3dde25ef3089'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:00.745555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1076/2140] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:00 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:00.752125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:00.755998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1065/2141] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:00 2022] DELETE /identity/v3/users/b1aa121d68ea4603a9aa3dde25ef3089 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:00.761590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:00.854773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbfeea-89fc-4be4-9f01-685fdb08428d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:00.967185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:00.968875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:00.969390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:00.969813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:00.972172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:00.976028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.009095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1077/2142] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:00 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:01.010523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.025901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ae341-eabc-44d8-8a3e-074b210aee0c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.031310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.032154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.033151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.033581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.035389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.038203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.057784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1066/2143] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:01.064396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.082003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8be44-56ac-4f7f-a7a9-96ad13869443 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.086793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.087467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.087735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.088009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.088762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.090515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.103045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1078/2144] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:01.112305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.146366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ee874-ca5b-4c93-b44b-2b3dd4dff10f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.153159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.155142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:01.156485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0f265-2ad7-4461-b5ab-5072b85623c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.157565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f265-2ad7-4461-b5ab-5072b85623c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.158101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f265-2ad7-4461-b5ab-5072b85623c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.158611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f265-2ad7-4461-b5ab-5072b85623c6 admin admin] PATH_INFO: `/v3/projects/54201078055b44fa83324467d553c895/users/db6e04247d7342d591bd62b605433ad4/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.189129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ffa31-8404-41f2-8e5e-4ef7133c0639 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.194439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.195861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.196139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.196587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] PATH_INFO: `/v3/projects/dccaff0cb7e94d2f933ac5130f9c356b` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.198109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccaff0cb7e94d2f933ac5130f9c356b)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.200287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.206090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0f265-2ad7-4461-b5ab-5072b85623c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54201078055b44fa83324467d553c895, user_id=db6e04247d7342d591bd62b605433ad4, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.210063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0f265-2ad7-4461-b5ab-5072b85623c6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.237497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1067/2145] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:01 2022] PUT /identity/v3/projects/54201078055b44fa83324467d553c895/users/db6e04247d7342d591bd62b605433ad4/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:01.247724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.248426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.248903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.290701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccaff0cb7e94d2f933ac5130f9c356b', 'request_id': 'req-a570b577-b703-4c2e-8d4d-28f44e1722e1'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.292797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] MFA Rules not processed for user `db6e04247d7342d591bd62b605433ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:01.294579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:01.303485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccaff0cb7e94d2f933ac5130f9c356b', 'request_id': 'req-a570b577-b703-4c2e-8d4d-28f44e1722e1'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.316613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccaff0cb7e94d2f933ac5130f9c356b', 'request_id': 'req-a570b577-b703-4c2e-8d4d-28f44e1722e1'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.329944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] The token cache is being invalidate because project dccaff0cb7e94d2f933ac5130f9c356b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:01.330233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.330897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.336128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1079/2146] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:01 2022] DELETE /identity/v3/projects/dccaff0cb7e94d2f933ac5130f9c356b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:01.338927 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.340026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:01.380490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1068/2147] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:01.386624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.388259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:01.423644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570b577-b703-4c2e-8d4d-28f44e1722e1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.429129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.430185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.430532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.430882 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] PATH_INFO: `/v3/users/b6eaaf036bdd4566bc48374ae481d7af` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.432176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6eaaf036bdd4566bc48374ae481d7af)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.434191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.466698 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b501a8-12b3-4e84-b0b9-27867647877d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.474730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.475567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.475939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.476281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.477162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.479187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.500547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6eaaf036bdd4566bc48374ae481d7af'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.501392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6eaaf036bdd4566bc48374ae481d7af'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.510235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6eaaf036bdd4566bc48374ae481d7af'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.512981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1069/2148] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:01 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:01.516820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.522038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6eaaf036bdd4566bc48374ae481d7af'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.522724 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6eaaf036bdd4566bc48374ae481d7af'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:01.545178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1080/2149] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:01 2022] DELETE /identity/v3/users/b6eaaf036bdd4566bc48374ae481d7af => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:01.594481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9a0b0-e6f8-4577-9412-cdcf108ccb8e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.671050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.671838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.672457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.672457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.673109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.675050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.700593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1070/2150] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:01 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:53:01.704556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.706260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:01.719557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85363321-fe81-4b0c-bca2-359c35141077 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.724792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.725494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.725777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.726049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.726775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.728352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.742388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1081/2151] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:01.744008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.760403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231c49a-e48b-41e6-930a-0be8a957929b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.772136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.773164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.773471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.773705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] PATH_INFO: `/v3/projects/ed5ff0edddca4d859d41c1e036e3db55/users/b5c8f3e2e6e9458291d99fbb74fb3760/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.794145 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed5ff0edddca4d859d41c1e036e3db55, user_id=b5c8f3e2e6e9458291d99fbb74fb3760, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.800301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.815940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1071/2152] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:01 2022] PUT /identity/v3/projects/ed5ff0edddca4d859d41c1e036e3db55/users/b5c8f3e2e6e9458291d99fbb74fb3760/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:01.820167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.830094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317dd1a-0b25-4d6f-af87-35feafeaf144 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.887342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.889297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:01.919964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0976336-057f-4a65-ab2c-08694097091a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.920931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.922937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.924505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.925330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.927264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.929484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.935241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.936500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.937290 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.937719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] PATH_INFO: `/v3/projects/5b8c1d7ac64f463b87f737a26f1a1f6e` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.939844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8c1d7ac64f463b87f737a26f1a1f6e)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.941611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.945431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1082/2153] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:01.946503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:01.958430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b1aa-afe2-494b-826b-16130268997d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:01.963539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c6161-2d81-45a0-8e16-25b19c0e460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:01.964816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c6161-2d81-45a0-8e16-25b19c0e460e admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:01.965008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c6161-2d81-45a0-8e16-25b19c0e460e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:01.965384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c6161-2d81-45a0-8e16-25b19c0e460e admin admin] PATH_INFO: `/v3/projects/ed5ff0edddca4d859d41c1e036e3db55/users/b5c8f3e2e6e9458291d99fbb74fb3760/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:01.967566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c6161-2d81-45a0-8e16-25b19c0e460e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed5ff0edddca4d859d41c1e036e3db55, user_id=b5c8f3e2e6e9458291d99fbb74fb3760, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:01.971045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c6161-2d81-45a0-8e16-25b19c0e460e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:01.986731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1083/2154] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:01 2022] PUT /identity/v3/projects/ed5ff0edddca4d859d41c1e036e3db55/users/b5c8f3e2e6e9458291d99fbb74fb3760/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:02.005620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.005984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.006350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.016594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8c1d7ac64f463b87f737a26f1a1f6e', 'request_id': 'req-a800694f-2666-413e-9f33-f091d514cdba'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:02.028015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8c1d7ac64f463b87f737a26f1a1f6e', 'request_id': 'req-a800694f-2666-413e-9f33-f091d514cdba'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:02.041292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8c1d7ac64f463b87f737a26f1a1f6e', 'request_id': 'req-a800694f-2666-413e-9f33-f091d514cdba'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:02.042193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] MFA Rules not processed for user `b5c8f3e2e6e9458291d99fbb74fb3760`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:02.043424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:02.058434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] The token cache is being invalidate because project 5b8c1d7ac64f463b87f737a26f1a1f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:02.059260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:02.060735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a800694f-2666-413e-9f33-f091d514cdba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:02.062363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1072/2155] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:01 2022] DELETE /identity/v3/projects/5b8c1d7ac64f463b87f737a26f1a1f6e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:02.069057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226534d-34cc-4db3-aac5-cd04aa4672ca None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.070201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226534d-34cc-4db3-aac5-cd04aa4672ca None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.070573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226534d-34cc-4db3-aac5-cd04aa4672ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.098092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b226534d-34cc-4db3-aac5-cd04aa4672ca None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:02.099996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b226534d-34cc-4db3-aac5-cd04aa4672ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:02.147730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1084/2156] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:53:02.155526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.156953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:02.192586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1073/2157] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:02.202024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.202507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.202932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.242871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:02.243711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa9714-0522-4ca9-b0f1-fda3670612cb None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.244055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:02.248357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.249698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.249698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.250082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.252084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:02.264672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1074/2158] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:02.268344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.290076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f6f9a-bf2f-40c1-9599-d8c7a14b6a1b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.297744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.299041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.299478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.299932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.301247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:53:02.301867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.304117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.323627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1075/2159] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:53:02.328479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.342886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.344475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.358219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff9b9-daaa-4354-aefe-b72598d74e31 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.359046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1085/2160] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:53:02.376256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.377101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.377481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.377862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.379325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.381797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.425575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1076/2161] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:02 2022] POST /identity/v3/projects => generated 397 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:02.426747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.462195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288fa10-30a8-4af7-859d-2db8a19f098f service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.586996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.587706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.587987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.588262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.589010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.591224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.639053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1086/2162] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:02 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:02.642795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.655988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6456f23-2067-4133-8d2c-d9365cad70be admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.666784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.667904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.668830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.669245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.670331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.673021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.685851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1077/2163] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:02.696668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.722959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177282b-df49-4036-a791-71e6a7ce41a1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.737374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae73d-e0ff-4fa7-ac89-de80f1df5423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.739558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae73d-e0ff-4fa7-ac89-de80f1df5423 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.740182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae73d-e0ff-4fa7-ac89-de80f1df5423 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.740733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae73d-e0ff-4fa7-ac89-de80f1df5423 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.742890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae73d-e0ff-4fa7-ac89-de80f1df5423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.746277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae73d-e0ff-4fa7-ac89-de80f1df5423 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.771346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1087/2164] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:02.773956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:02.791260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e632-c92b-48e0-9580-83e35c317f46 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:02.804589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:02.808001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.808454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.808917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] PATH_INFO: `/v3/projects/534a32ba6b574a1ba7a3e909bc98cc33/users/590c419d2c594138a0ea5d6a4a4bb7ed/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.829553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] RBAC: Authorizing `identity:create_grant(project_id=534a32ba6b574a1ba7a3e909bc98cc33, user_id=590c419d2c594138a0ea5d6a4a4bb7ed, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:02.833451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:02.875617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1078/2165] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:02 2022] PUT /identity/v3/projects/534a32ba6b574a1ba7a3e909bc98cc33/users/590c419d2c594138a0ea5d6a4a4bb7ed/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:02.880352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536cce4-5ee0-4368-b493-cab9b56a47ca None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:02.880667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536cce4-5ee0-4368-b493-cab9b56a47ca None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:02.881112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536cce4-5ee0-4368-b493-cab9b56a47ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:02.921135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4536cce4-5ee0-4368-b493-cab9b56a47ca None None] MFA Rules not processed for user `590c419d2c594138a0ea5d6a4a4bb7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:02.923389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4536cce4-5ee0-4368-b493-cab9b56a47ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:03.005598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1088/2166] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:03.016367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:03.114708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc87447-9c66-461f-a084-30cb4e1e349c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:03.119521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:03.120213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:03.120552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:03.120861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:03.141405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:03.143520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:03.156427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1079/2167] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:53:04.224138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca7128-ef41-4f30-8924-2d8b440aab21 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:04.224691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca7128-ef41-4f30-8924-2d8b440aab21 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:04.224691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca7128-ef41-4f30-8924-2d8b440aab21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:04.241783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-06ca7128-ef41-4f30-8924-2d8b440aab21 None None] MFA Rules not processed for user `18a2a873aeae41de8b25c2095c645479`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:04.242960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-06ca7128-ef41-4f30-8924-2d8b440aab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:04.296298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1089/2168] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:09.169614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:09.179786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cec43-8ea9-4ecd-afbb-b44c0fbde54b service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:09.183961 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:09.184407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:09.184789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:09.185015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:09.216984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:09.217514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:09.217605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:09.236403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:09.238545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:09.259985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:09.261536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:09.272332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1080/2169] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:53:09.308014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1090/2170] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:53:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:53:09.320897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:09.322570 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:09.408761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4bdaf-538d-4a26-8092-73ce7c3af72b service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:09.415301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3710f2-0ea5-441c-b320-61395994c6d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:09.416037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3710f2-0ea5-441c-b320-61395994c6d9 service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:09.416457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3710f2-0ea5-441c-b320-61395994c6d9 service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:09.416832 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3710f2-0ea5-441c-b320-61395994c6d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:09.428732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3710f2-0ea5-441c-b320-61395994c6d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:09.434478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3710f2-0ea5-441c-b320-61395994c6d9 service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:09.453482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1081/2171] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:53:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:53:31.108009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:31.211102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c36e8-25d0-4cd8-b7a8-0789cbc3a239 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:31.215587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:31.216445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:31.216969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:31.217248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] PATH_INFO: `/v3/users/590c419d2c594138a0ea5d6a4a4bb7ed` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:31.218430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=590c419d2c594138a0ea5d6a4a4bb7ed)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:31.220319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:31.262970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590c419d2c594138a0ea5d6a4a4bb7ed'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.263692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590c419d2c594138a0ea5d6a4a4bb7ed'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.270738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590c419d2c594138a0ea5d6a4a4bb7ed'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.278326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590c419d2c594138a0ea5d6a4a4bb7ed'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.278929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590c419d2c594138a0ea5d6a4a4bb7ed'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.296041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1091/2172] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:31 2022] DELETE /identity/v3/users/590c419d2c594138a0ea5d6a4a4bb7ed => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:31.300577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:31.301130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:31.301130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:31.348141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:31.349366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:31.404004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1082/2173] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:31.430372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:31.431767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:31.523485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaca97-a879-4282-9644-d9bf0b6cc14a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:31.527460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:31.528102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:31.528383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:31.528684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:31.539438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:31.541105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:31.551427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1092/2174] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:53:31.819132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:31.821270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:31.833766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e37bf-cd96-4fb5-baeb-f967843697ca None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:31.838464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:31.839374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:31.839727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:31.840080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] PATH_INFO: `/v3/projects/534a32ba6b574a1ba7a3e909bc98cc33` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:31.841288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=534a32ba6b574a1ba7a3e909bc98cc33)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:31.843175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:31.887439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534a32ba6b574a1ba7a3e909bc98cc33', 'request_id': 'req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.892569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534a32ba6b574a1ba7a3e909bc98cc33', 'request_id': 'req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.898469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534a32ba6b574a1ba7a3e909bc98cc33', 'request_id': 'req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.910635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] The token cache is being invalidate because project 534a32ba6b574a1ba7a3e909bc98cc33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:31.910915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.911546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ace60f2e-e3c6-4fef-8f9a-064e66fa83bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:31.913123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1083/2175] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:31 2022] DELETE /identity/v3/projects/534a32ba6b574a1ba7a3e909bc98cc33 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:49.617766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:49.620685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:49.637287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5190222-cf9f-4f52-a780-0b8b385ab7dc service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:49.722020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:49.723447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:49.723909 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:49.724393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] PATH_INFO: `/v3/users/db6e04247d7342d591bd62b605433ad4` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:49.726590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6e04247d7342d591bd62b605433ad4)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:49.730304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:49.800048 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6e04247d7342d591bd62b605433ad4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:49.800963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6e04247d7342d591bd62b605433ad4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:49.808346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6e04247d7342d591bd62b605433ad4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:49.816581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6e04247d7342d591bd62b605433ad4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:49.817271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6e04247d7342d591bd62b605433ad4'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:49.839648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1093/2176] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:49 2022] DELETE /identity/v3/users/db6e04247d7342d591bd62b605433ad4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:49.841566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:49.842350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:49.842634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:49.870830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:49.872013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:49.933971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1084/2177] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:53:49.944222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:49.945683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:50.000047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472145-5aee-41a7-9c0f-7426d96de48b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:50.004331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:50.005003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:50.005282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:50.005555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:50.014301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:50.015704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:50.024843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1094/2178] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:53:50.299573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:50.301712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:50.323469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fe0e3-d882-4eb8-ab95-35462da94843 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:50.328955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:50.329809 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:50.330086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:50.330356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] PATH_INFO: `/v3/projects/54201078055b44fa83324467d553c895` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:50.331545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54201078055b44fa83324467d553c895)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:50.333501 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:50.378434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54201078055b44fa83324467d553c895', 'request_id': 'req-9be10df6-3dfc-4e57-b177-7eee641831d3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.383715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54201078055b44fa83324467d553c895', 'request_id': 'req-9be10df6-3dfc-4e57-b177-7eee641831d3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.389686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54201078055b44fa83324467d553c895', 'request_id': 'req-9be10df6-3dfc-4e57-b177-7eee641831d3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.401034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] The token cache is being invalidate because project 54201078055b44fa83324467d553c895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:50.401316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.401919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.403413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1085/2179] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:50 2022] DELETE /identity/v3/projects/54201078055b44fa83324467d553c895 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:50.407361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:50.417317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a5bf-28b1-41c6-b6fc-0aff7864478d service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:50.473293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:50.474065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:50.474347 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:50.474618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] PATH_INFO: `/v3/users/b5c8f3e2e6e9458291d99fbb74fb3760` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:50.475780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c8f3e2e6e9458291d99fbb74fb3760)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:50.477774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:50.520756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c8f3e2e6e9458291d99fbb74fb3760'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.521380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c8f3e2e6e9458291d99fbb74fb3760'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.527787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c8f3e2e6e9458291d99fbb74fb3760'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.534734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c8f3e2e6e9458291d99fbb74fb3760'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.535411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d0015c04-3f09-4a0c-aefb-e32bd4a7954d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c8f3e2e6e9458291d99fbb74fb3760'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.552254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1095/2180] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:50 2022] DELETE /identity/v3/users/b5c8f3e2e6e9458291d99fbb74fb3760 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:50.832838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:50.834740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:50.871700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be10df6-3dfc-4e57-b177-7eee641831d3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:50.933036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:50.933886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:50.934163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:50.934436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] PATH_INFO: `/v3/projects/ed5ff0edddca4d859d41c1e036e3db55` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:50.935624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed5ff0edddca4d859d41c1e036e3db55)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:50.937562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:50.993072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed5ff0edddca4d859d41c1e036e3db55', 'request_id': 'req-7bdeb905-f931-4968-b38f-e4e8769bbdb3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:50.998507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed5ff0edddca4d859d41c1e036e3db55', 'request_id': 'req-7bdeb905-f931-4968-b38f-e4e8769bbdb3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:51.006138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed5ff0edddca4d859d41c1e036e3db55', 'request_id': 'req-7bdeb905-f931-4968-b38f-e4e8769bbdb3'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:51.018119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] The token cache is being invalidate because project ed5ff0edddca4d859d41c1e036e3db55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:51.018955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:51.019582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7bdeb905-f931-4968-b38f-e4e8769bbdb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:51.021122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1086/2181] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:50 2022] DELETE /identity/v3/projects/ed5ff0edddca4d859d41c1e036e3db55 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:51.025914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.026353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.026756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.074352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:51.075556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:51.138090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1096/2182] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:51.146829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.147119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.147395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.165718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:51.166883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:51.174620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1087/2183] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:51.178719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.188141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101f92a-5971-4e11-8961-599fa6746c4e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.193309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.193956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.194239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.194516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.195236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:53:51.195542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.197160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.209106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1097/2184] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:53:51.213137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.222047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85240332-90ba-454e-850d-6d8c710b8857 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.226619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.227245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.227521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.227788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.228465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.230378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.255525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1088/2185] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/projects => generated 393 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:51.259464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.269871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ed9eb-7391-4525-950a-b5969faf7f56 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.324796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.325457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.325735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.326005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.326712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.328617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.351209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1098/2186] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:51.353146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.363783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daa46d-2dec-4012-95ed-3c6740f77e5f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.368577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.369264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.369539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.369812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.370863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.372120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.383596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1089/2187] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:51.387554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.397414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125325c-0960-4636-97a5-101b7cdeb8a5 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.402035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13960692-a16e-47ff-a6f6-b4d5272c5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.402682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13960692-a16e-47ff-a6f6-b4d5272c5ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.402979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13960692-a16e-47ff-a6f6-b4d5272c5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.403236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13960692-a16e-47ff-a6f6-b4d5272c5ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.403952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13960692-a16e-47ff-a6f6-b4d5272c5ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.405572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13960692-a16e-47ff-a6f6-b4d5272c5ec0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.419924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1099/2188] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:53:51.421248 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.433979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971284-7dfe-42a3-9dee-da42bffc0455 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.441234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.442272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.443257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.443530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] PATH_INFO: `/v3/projects/9c0925139c744784affc77f5c8e205e6/users/fdbf79964f9645de95fdb4436cc3e6d9/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.459255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0925139c744784affc77f5c8e205e6, user_id=fdbf79964f9645de95fdb4436cc3e6d9, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.461913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.474322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1090/2189] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:51 2022] PUT /identity/v3/projects/9c0925139c744784affc77f5c8e205e6/users/fdbf79964f9645de95fdb4436cc3e6d9/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:51.485341 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.485631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.485906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.512232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] MFA Rules not processed for user `fdbf79964f9645de95fdb4436cc3e6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:51.513404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:51.573630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1100/2190] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:51.578056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.636890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238a4aa-fbd9-47bd-9022-ad2c0ebc6340 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.641790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.642432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.642713 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.642986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.643718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.645646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.670232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1091/2191] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/projects => generated 393 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:53:51.674180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.687597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091788a0-20e9-465b-a84c-b1022ba34601 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.746206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.746875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.747161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.747427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.748154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.750201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.768793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1101/2192] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:53:51.772578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.782621 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227586ef-b7b5-4b48-947a-6aaf52ae8d66 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.788092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.788778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.789068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.789347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.790069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.791654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.803446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1092/2193] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:51.807464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.817194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0e2db-7a0f-4c17-b4ca-f96e45070089 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.821767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.822628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.822911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.823188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] PATH_INFO: `/v3/projects/985d6fbd7abb4d6f808130f044680d38/users/74fe9413b00a465ba65e18f647c867e5/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.837277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985d6fbd7abb4d6f808130f044680d38, user_id=74fe9413b00a465ba65e18f647c867e5, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.840120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.853577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1102/2194] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:51 2022] PUT /identity/v3/projects/985d6fbd7abb4d6f808130f044680d38/users/74fe9413b00a465ba65e18f647c867e5/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:51.857406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.867599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a1c6-842a-4180-aff4-d2d5146bc2e2 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.921157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d8efb-6165-418e-a45d-2211839d3076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.921844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d8efb-6165-418e-a45d-2211839d3076 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.922127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d8efb-6165-418e-a45d-2211839d3076 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.922401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d8efb-6165-418e-a45d-2211839d3076 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.923121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d8efb-6165-418e-a45d-2211839d3076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.925389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d8efb-6165-418e-a45d-2211839d3076 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.934884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1093/2195] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:51.938955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:51.948992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5f669-facc-46f8-9f42-b90f9c0c4374 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:51.953527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:51.954389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.954692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.954967 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] PATH_INFO: `/v3/projects/985d6fbd7abb4d6f808130f044680d38/users/74fe9413b00a465ba65e18f647c867e5/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:51.956994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985d6fbd7abb4d6f808130f044680d38, user_id=74fe9413b00a465ba65e18f647c867e5, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:51.959607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:51.969658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1103/2196] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:51 2022] PUT /identity/v3/projects/985d6fbd7abb4d6f808130f044680d38/users/74fe9413b00a465ba65e18f647c867e5/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:51.980090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:51.980376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:51.980669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:52.005965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] MFA Rules not processed for user `74fe9413b00a465ba65e18f647c867e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:52.007131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:52.070826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1094/2197] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:52.082359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:52.084124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:52.163501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd34a39-e7a2-4190-8b41-41912df0e7e6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:52.167180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:52.167817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:52.168096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:52.168370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:52.177140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:52.178575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:52.188001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1104/2198] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:53:52.233480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:52.234861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:52.246555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763c2c9-b8c5-4409-820b-c7a176e9ef38 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:52.250135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec9261-bbb7-4547-8146-c7ded0249562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:52.250765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec9261-bbb7-4547-8146-c7ded0249562 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:52.251052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec9261-bbb7-4547-8146-c7ded0249562 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:52.251315 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec9261-bbb7-4547-8146-c7ded0249562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:52.308746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec9261-bbb7-4547-8146-c7ded0249562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:52.310266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec9261-bbb7-4547-8146-c7ded0249562 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:52.319963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1095/2199] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:53:55.760262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:55.821605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550a133-5e96-4caf-82ed-0848afb2ef42 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:55.826126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:55.826985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:55.827271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:55.827544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] PATH_INFO: `/v3/users/fdbf79964f9645de95fdb4436cc3e6d9` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:55.828756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbf79964f9645de95fdb4436cc3e6d9)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:55.831198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:55.877567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbf79964f9645de95fdb4436cc3e6d9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:55.878279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbf79964f9645de95fdb4436cc3e6d9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:55.885008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbf79964f9645de95fdb4436cc3e6d9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:55.893877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbf79964f9645de95fdb4436cc3e6d9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:55.894772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbf79964f9645de95fdb4436cc3e6d9'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:55.914485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1105/2200] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:55 2022] DELETE /identity/v3/users/fdbf79964f9645de95fdb4436cc3e6d9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:55.919118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:55.919629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:55.919745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:55.936505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:55.937810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:55.999321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1096/2201] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:56.008916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:56.010328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:56.068804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a21c9-1f04-409e-8849-e4458849aa2a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:56.072438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead049c-2986-4228-9aaf-23cb3eff7d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:56.073120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead049c-2986-4228-9aaf-23cb3eff7d34 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:56.073388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead049c-2986-4228-9aaf-23cb3eff7d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:56.074043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead049c-2986-4228-9aaf-23cb3eff7d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:56.083235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead049c-2986-4228-9aaf-23cb3eff7d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:56.084752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead049c-2986-4228-9aaf-23cb3eff7d34 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:56.094323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1106/2202] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:53:56.363071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:56.364504 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:56.367851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:56.368232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:56.368748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:56.379024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442c6ef-9f74-4ac0-b52c-6325502ff1db None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:56.383879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:56.384979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:56.385412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:56.385841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] PATH_INFO: `/v3/projects/9c0925139c744784affc77f5c8e205e6` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:56.387053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] MFA Rules not processed for user `3b39f00235c34612b591cd2d974d9969`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:56.387191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0925139c744784affc77f5c8e205e6)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:56.388259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:56.389316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:56.437443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0925139c744784affc77f5c8e205e6', 'request_id': 'req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.444380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0925139c744784affc77f5c8e205e6', 'request_id': 'req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.445234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1107/2203] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:53:56.454838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0925139c744784affc77f5c8e205e6', 'request_id': 'req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.468048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] The token cache is being invalidate because project 9c0925139c744784affc77f5c8e205e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:56.468333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.469025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.470790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1097/2204] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:56 2022] DELETE /identity/v3/projects/9c0925139c744784affc77f5c8e205e6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:56.474770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:56.485283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0602db4-b908-4b3b-8fbf-3b04058bd26b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:56.540324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:56.541241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:56.541521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:56.541815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] PATH_INFO: `/v3/users/74fe9413b00a465ba65e18f647c867e5` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:56.543187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fe9413b00a465ba65e18f647c867e5)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:56.545139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:56.592517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fe9413b00a465ba65e18f647c867e5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.593398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fe9413b00a465ba65e18f647c867e5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.600059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fe9413b00a465ba65e18f647c867e5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.607598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fe9413b00a465ba65e18f647c867e5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.608552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d7216888-4bc1-4ead-bf30-07849bea02b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fe9413b00a465ba65e18f647c867e5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:56.625925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1108/2205] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:53:56 2022] DELETE /identity/v3/users/74fe9413b00a465ba65e18f647c867e5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:57.059856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.061508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:57.074893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a935cfa-ac20-41a8-a3d9-92ce3b008c62 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.135343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.136262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.136557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.136862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] PATH_INFO: `/v3/projects/985d6fbd7abb4d6f808130f044680d38` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.138331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985d6fbd7abb4d6f808130f044680d38)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.140457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.193419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985d6fbd7abb4d6f808130f044680d38', 'request_id': 'req-925be769-da1b-4412-b885-46bef9fbf2f7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:57.198958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985d6fbd7abb4d6f808130f044680d38', 'request_id': 'req-925be769-da1b-4412-b885-46bef9fbf2f7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:57.205852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985d6fbd7abb4d6f808130f044680d38', 'request_id': 'req-925be769-da1b-4412-b885-46bef9fbf2f7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:57.221156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] The token cache is being invalidate because project 985d6fbd7abb4d6f808130f044680d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:53:57.221754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:57.222638 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-925be769-da1b-4412-b885-46bef9fbf2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:53:57.230808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1098/2206] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:53:57 2022] DELETE /identity/v3/projects/985d6fbd7abb4d6f808130f044680d38 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:57.238556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.239040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.239128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.258899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:57.260090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:57.319622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1109/2207] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:57.327476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.327766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.328041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.346537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:57.347725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:57.355883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1099/2208] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:57.360000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.370976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89969eff-e064-41ca-8860-a2ba0e651a47 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.375934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.376591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.376902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.377175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.378468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:53:57.378770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.380396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.392219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1110/2209] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:53:57.396643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.405671 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ba84c-de5f-4edd-9765-31d4f2a8dfb4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.411820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.412481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.412792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.413069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.413820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.415661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.443078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1100/2210] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:57.447288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.457111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb6e77-6849-4f80-8a1e-8f45e01b9fe9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.513926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.514589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.514870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.515144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.515822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.517761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.536916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1111/2211] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:53:57.541754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.552667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89fc7b-8173-4907-8ec2-432e64af54e3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.557342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.557942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.558222 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.558497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.559218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.560847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.574927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1101/2212] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:57.576191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.586272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3122-a039-468d-adda-09d49fad37a5 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.591328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea517532-9104-46a7-8321-2f159a5835a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.591982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea517532-9104-46a7-8321-2f159a5835a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.592264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea517532-9104-46a7-8321-2f159a5835a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.592540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea517532-9104-46a7-8321-2f159a5835a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.593326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea517532-9104-46a7-8321-2f159a5835a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.594906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea517532-9104-46a7-8321-2f159a5835a1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.605507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1112/2213] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:57.609557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.620876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe567ac7-e383-4254-b830-8151cb220278 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.626058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.626927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.627205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.628067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] PATH_INFO: `/v3/projects/b07fec7ca44c4f17a2b8951165b29e0e/users/64bf1147ad1d4f9ebc0be8850cce06ff/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.642295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07fec7ca44c4f17a2b8951165b29e0e, user_id=64bf1147ad1d4f9ebc0be8850cce06ff, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.644926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.657835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1102/2214] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:57 2022] PUT /identity/v3/projects/b07fec7ca44c4f17a2b8951165b29e0e/users/64bf1147ad1d4f9ebc0be8850cce06ff/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:57.669310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.669622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.669903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.696488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] MFA Rules not processed for user `64bf1147ad1d4f9ebc0be8850cce06ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:57.697702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:57.762597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1113/2215] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:57.767308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.841103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a40dd-177f-49a1-9713-bc79652537b7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.845767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.846913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.847192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.847467 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.848294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.850226 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.875396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1103/2216] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:57.879494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.889844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41943-da54-458a-b4cc-fe610ec1bd89 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.944694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.945681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.946094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.946376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.947070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.949017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:57.968554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1114/2217] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:53:57 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:53:57.973071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:57.983218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2504f-296f-4ff3-ab09-2f58f3a8be4e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:57.988370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:57.989046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:57.989334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:57.989609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:57.990465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:57.992063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:58.002250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1104/2218] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:58.006030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:58.015564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529a49d-4776-43bd-9978-46fbe87dadd0 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:58.020079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:58.020965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:58.021246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:58.021519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] PATH_INFO: `/v3/projects/9b4d5211d78b4de98868f6594e3c532a/users/f14d861af9eb4dc784e57fc3beca7421/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:58.036019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4d5211d78b4de98868f6594e3c532a, user_id=f14d861af9eb4dc784e57fc3beca7421, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:58.038654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:58.051041 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1115/2219] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:58 2022] PUT /identity/v3/projects/9b4d5211d78b4de98868f6594e3c532a/users/f14d861af9eb4dc784e57fc3beca7421/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:53:58.054831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:58.066927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf485ff6-1518-4f00-925f-e8d0ca595c49 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:58.123056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbbab8-cb55-4116-b46f-69eeeb69ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:58.123715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbbab8-cb55-4116-b46f-69eeeb69ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:58.123999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbbab8-cb55-4116-b46f-69eeeb69ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:58.124277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbbab8-cb55-4116-b46f-69eeeb69ac6b admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:58.125031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbbab8-cb55-4116-b46f-69eeeb69ac6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:58.126618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbbab8-cb55-4116-b46f-69eeeb69ac6b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:58.140409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1105/2220] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:53:58.141916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:58.154381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae15d4-3344-44cb-bbf8-8fce7a969dc4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:58.159962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:58.160864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:58.161142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:58.161413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] PATH_INFO: `/v3/projects/9b4d5211d78b4de98868f6594e3c532a/users/f14d861af9eb4dc784e57fc3beca7421/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:58.163352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4d5211d78b4de98868f6594e3c532a, user_id=f14d861af9eb4dc784e57fc3beca7421, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:58.166039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:58.176085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1116/2221] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:53:58 2022] PUT /identity/v3/projects/9b4d5211d78b4de98868f6594e3c532a/users/f14d861af9eb4dc784e57fc3beca7421/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:53:58.186382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:58.186672 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:58.186986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:58.213641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] MFA Rules not processed for user `f14d861af9eb4dc784e57fc3beca7421`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:53:58.214857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:58.289667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1106/2222] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:53:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:53:58.300992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:53:58.302893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:53:58.426407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7025-c319-41ef-aa71-6621f8be0c01 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:53:58.430516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e5473-3412-4d57-920e-56e685be251a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:53:58.431165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e5473-3412-4d57-920e-56e685be251a service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:53:58.431451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e5473-3412-4d57-920e-56e685be251a service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:53:58.431726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e5473-3412-4d57-920e-56e685be251a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:53:58.490813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e5473-3412-4d57-920e-56e685be251a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:53:58.492409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e5473-3412-4d57-920e-56e685be251a service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:53:58.504661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1117/2223] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:53:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:54:01.055346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:01.122555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07825815-ab6b-42eb-b166-ddf7a3560127 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:01.127443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:01.128371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:01.128749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:01.129079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] PATH_INFO: `/v3/users/64bf1147ad1d4f9ebc0be8850cce06ff` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:01.130329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=64bf1147ad1d4f9ebc0be8850cce06ff)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:01.132426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:01.185643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64bf1147ad1d4f9ebc0be8850cce06ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.193282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64bf1147ad1d4f9ebc0be8850cce06ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.194076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64bf1147ad1d4f9ebc0be8850cce06ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.204091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64bf1147ad1d4f9ebc0be8850cce06ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.205121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64bf1147ad1d4f9ebc0be8850cce06ff'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.232753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1107/2224] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:54:01 2022] DELETE /identity/v3/users/64bf1147ad1d4f9ebc0be8850cce06ff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:01.237297 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:01.237827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:01.238064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:01.256099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:01.257516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:01.304830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:01.306460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:01.324198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1118/2225] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:54:01.333866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:01.335216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:01.365618 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be0ad9-b2c5-4c59-bdd1-aa161b016ace admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:01.369263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:01.369938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:01.370184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:01.370453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:01.372227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:01.389370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8b30d-6b46-4a2b-9037-9141d1110144 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:01.393802 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:01.394441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:01.394720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:01.394992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:01.403876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:01.405332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:01.415447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1119/2226] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:54:01.433350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:01.434798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:01.444307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1108/2227] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:54:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:54:01.689073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:01.691051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:01.708773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409489-98c6-4df2-b9bd-80ae3f7e6e19 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:01.713333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:01.714433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:01.714751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:01.715090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] PATH_INFO: `/v3/projects/b07fec7ca44c4f17a2b8951165b29e0e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:01.716773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07fec7ca44c4f17a2b8951165b29e0e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:01.719111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:01.775074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07fec7ca44c4f17a2b8951165b29e0e', 'request_id': 'req-42777c1f-23d3-45b1-bd7e-33f285cb0f92'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.780741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07fec7ca44c4f17a2b8951165b29e0e', 'request_id': 'req-42777c1f-23d3-45b1-bd7e-33f285cb0f92'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.792012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07fec7ca44c4f17a2b8951165b29e0e', 'request_id': 'req-42777c1f-23d3-45b1-bd7e-33f285cb0f92'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.805332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] The token cache is being invalidate because project b07fec7ca44c4f17a2b8951165b29e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:54:01.805435 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.806061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.810703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1120/2228] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:54:01 2022] DELETE /identity/v3/projects/b07fec7ca44c4f17a2b8951165b29e0e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:01.814514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:01.825534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf42adc-6523-4d4e-9747-d819ee180d73 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:01.887163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:01.888034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:01.888311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:01.888580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] PATH_INFO: `/v3/users/f14d861af9eb4dc784e57fc3beca7421` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:01.889795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f14d861af9eb4dc784e57fc3beca7421)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:01.891692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:01.941822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f14d861af9eb4dc784e57fc3beca7421'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.948879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f14d861af9eb4dc784e57fc3beca7421'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.949550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f14d861af9eb4dc784e57fc3beca7421'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.956971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f14d861af9eb4dc784e57fc3beca7421'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.957572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-600c2403-752b-4eb6-9149-45c300bd9f1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f14d861af9eb4dc784e57fc3beca7421'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:01.975829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1109/2229] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:54:01 2022] DELETE /identity/v3/users/f14d861af9eb4dc784e57fc3beca7421 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:02.253277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.254733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:02.267812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777c1f-23d3-45b1-bd7e-33f285cb0f92 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.327847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.328923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.329278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.329716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] PATH_INFO: `/v3/projects/9b4d5211d78b4de98868f6594e3c532a` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.331653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4d5211d78b4de98868f6594e3c532a)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.333693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.379807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4d5211d78b4de98868f6594e3c532a', 'request_id': 'req-87f3ae24-b6bb-4209-9afe-86eb37581276'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:02.385137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4d5211d78b4de98868f6594e3c532a', 'request_id': 'req-87f3ae24-b6bb-4209-9afe-86eb37581276'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:02.394077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4d5211d78b4de98868f6594e3c532a', 'request_id': 'req-87f3ae24-b6bb-4209-9afe-86eb37581276'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:02.405065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] The token cache is being invalidate because project 9b4d5211d78b4de98868f6594e3c532a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:54:02.405350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:02.406012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-87f3ae24-b6bb-4209-9afe-86eb37581276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:02.407596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1121/2230] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:54:02 2022] DELETE /identity/v3/projects/9b4d5211d78b4de98868f6594e3c532a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:02.412691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.413505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.413788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.432419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:02.433645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:02.491846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1110/2231] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:54:02.499606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.499892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.500165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.519203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:02.520430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:02.528208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1122/2232] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:54:02.533916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.543885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39a9c9-10e5-4b43-8593-0e4041259155 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.548702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.549391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.549667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.549938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.550659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:54:02.550962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.552535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.563704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1111/2233] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:54:02.568376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.577606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d82f4e-ecf2-4993-b4b3-091aaaa3c6e8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.582871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.583516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.583799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.584071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.584799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.586686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.614980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1123/2234] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:54:02 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:54:02.616350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.626136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d59274-664e-4fd3-bf4d-b4ed3b05ea65 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.680980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.681647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.681925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.682197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.682892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.684814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.705432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1112/2235] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:54:02 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:54:02.709284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.719390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c49818-4031-4a2f-b759-128b4841277e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.724087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.724847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.725201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.725537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.726336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.727895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.738759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1124/2236] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:54:02.742596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.753649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03408a6-bf39-4e39-a80c-6cf401d64c31 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.758313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506f2a6-c5c3-4b98-abdc-11b1bd535c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.758995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506f2a6-c5c3-4b98-abdc-11b1bd535c4c admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.759276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506f2a6-c5c3-4b98-abdc-11b1bd535c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.759548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506f2a6-c5c3-4b98-abdc-11b1bd535c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.760268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506f2a6-c5c3-4b98-abdc-11b1bd535c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.761900 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506f2a6-c5c3-4b98-abdc-11b1bd535c4c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.772188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1113/2237] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:54:02.776060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.786424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c10805-160c-4aac-b05c-30cc81d4ab1a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:02.791430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:02.792485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.792819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.793097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] PATH_INFO: `/v3/projects/7be4d756763e45079901ee8c3a63d632/users/a5501089299641ca941d4ecb927019ce/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.808980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be4d756763e45079901ee8c3a63d632, user_id=a5501089299641ca941d4ecb927019ce, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:02.811606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:02.825994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1125/2238] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:54:02 2022] PUT /identity/v3/projects/7be4d756763e45079901ee8c3a63d632/users/a5501089299641ca941d4ecb927019ce/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:02.835978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:02.836327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:02.836704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:02.863910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] MFA Rules not processed for user `a5501089299641ca941d4ecb927019ce`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:02.865171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:02.944416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1114/2239] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:54:02.955888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:02.957591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:03.035677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ac345-9280-43c4-bdaa-4c0b6749ef47 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:03.039634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e261c65-e512-46cc-8fc1-99ed59587f98 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:03.040270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261c65-e512-46cc-8fc1-99ed59587f98 service glance] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:03.040550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261c65-e512-46cc-8fc1-99ed59587f98 service glance] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:03.040852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261c65-e512-46cc-8fc1-99ed59587f98 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:03.050032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e261c65-e512-46cc-8fc1-99ed59587f98 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:03.051613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e261c65-e512-46cc-8fc1-99ed59587f98 service glance] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:03.062164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1126/2240] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:54:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:54:42.180920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:42.241394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c074cd-a3b6-49b7-bec4-51904cdcdecf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:42.245925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:42.246830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.247133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.247409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] PATH_INFO: `/v3/users/a5501089299641ca941d4ecb927019ce` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.248566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5501089299641ca941d4ecb927019ce)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:42.250480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:42.294349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5501089299641ca941d4ecb927019ce'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.301037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5501089299641ca941d4ecb927019ce'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.301692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5501089299641ca941d4ecb927019ce'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.308719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5501089299641ca941d4ecb927019ce'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.309354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5501089299641ca941d4ecb927019ce'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.325550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1115/2241] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:54:42 2022] DELETE /identity/v3/users/a5501089299641ca941d4ecb927019ce => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:42.329969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.330491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.330491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.349450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:42.350639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:42.405834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1127/2242] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:54:42.416177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:42.417560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:42.470878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18a99e-be78-4e30-803c-5ed9ecf236df admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:42.475111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49fed7-639f-4870-a4ae-53945ad5b5f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:42.475734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49fed7-639f-4870-a4ae-53945ad5b5f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.476010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49fed7-639f-4870-a4ae-53945ad5b5f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.476281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49fed7-639f-4870-a4ae-53945ad5b5f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.485040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49fed7-639f-4870-a4ae-53945ad5b5f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:42.486462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49fed7-639f-4870-a4ae-53945ad5b5f7 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:42.495792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1116/2243] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:54:42.761231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:42.762652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:42.775110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41395ad5-7ba7-4761-b21f-6e14f70b0298 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:42.780197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:42.781128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.781368 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.781639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] PATH_INFO: `/v3/projects/7be4d756763e45079901ee8c3a63d632` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.782862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be4d756763e45079901ee8c3a63d632)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:42.785232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:42.832286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be4d756763e45079901ee8c3a63d632', 'request_id': 'req-c127cfe4-19ee-4191-a42c-0639a3da5dfe'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.837442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be4d756763e45079901ee8c3a63d632', 'request_id': 'req-c127cfe4-19ee-4191-a42c-0639a3da5dfe'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.845919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be4d756763e45079901ee8c3a63d632', 'request_id': 'req-c127cfe4-19ee-4191-a42c-0639a3da5dfe'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.856980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] The token cache is being invalidate because project 7be4d756763e45079901ee8c3a63d632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:54:42.857262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.857882 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-c127cfe4-19ee-4191-a42c-0639a3da5dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:54:42.859408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1128/2244] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:54:42 2022] DELETE /identity/v3/projects/7be4d756763e45079901ee8c3a63d632 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:54:42.864160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.864519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.864879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.882299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:42.883464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:42.938235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1117/2245] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:54:42.946505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.946788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.947063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.966749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:42.967932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:42.975962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1129/2246] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:54:42.980103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:42.989488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b675c-f2cb-4e62-8285-e2811d3dbed5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:42.994043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:42.994683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:42.994964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:42.995236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:42.995949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:54:42.996255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:42.997872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.009545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1118/2247] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:54:43.013434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:43.022750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9db36-b9f2-488b-adce-2f34a0aff9d9 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:43.027327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:43.027952 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.028227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.028497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.029220 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:43.031120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.056707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1130/2248] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:54:43 2022] POST /identity/v3/projects => generated 389 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:54:43.060134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:43.070653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc6d8d-ef7a-4d6c-86cc-f0c429d040e0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:43.123451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:43.124099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.124375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.124653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.125409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:43.127322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.148188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1119/2249] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:54:43 2022] POST /identity/v3/users => generated 295 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:54:43.152401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:43.162806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c9420-0f06-47a6-9b89-7bc2a44f2291 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:43.167456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:43.168100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.168380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.168682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.169402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:43.171005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.181556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1131/2250] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:54:43.185454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:43.195812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e73d-3490-4c31-9165-bbf21c23d32e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:43.200353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20351c20-ebd4-4b21-92cf-de7990c31eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:43.201029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20351c20-ebd4-4b21-92cf-de7990c31eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.201307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20351c20-ebd4-4b21-92cf-de7990c31eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.201579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20351c20-ebd4-4b21-92cf-de7990c31eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.202317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20351c20-ebd4-4b21-92cf-de7990c31eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:43.203887 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20351c20-ebd4-4b21-92cf-de7990c31eb0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.214010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1120/2251] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:54:43.217791 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:43.227926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0e5c2-2925-4088-8755-64b7b97e37b7 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:43.232506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:43.233391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.233669 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.233937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] PATH_INFO: `/v3/projects/21e82b4b2fb04a989adfdc1f168850f2/users/ee8a62866c2f4b838f1691c24366ca57/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.247649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e82b4b2fb04a989adfdc1f168850f2, user_id=ee8a62866c2f4b838f1691c24366ca57, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:43.250329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.262869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1132/2252] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:54:43 2022] PUT /identity/v3/projects/21e82b4b2fb04a989adfdc1f168850f2/users/ee8a62866c2f4b838f1691c24366ca57/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:54:43.274147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.274437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.274735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.300808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] MFA Rules not processed for user `ee8a62866c2f4b838f1691c24366ca57`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:54:43.301998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:43.363442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1121/2253] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:54:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:54:43.373895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:54:43.375681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:54:43.431439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314be5e8-dfba-4d1a-84d8-2ab4e9a9b5a9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:54:43.440910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:54:43.441554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:54:43.441829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:54:43.442097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:54:43.450667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:54:43.452070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:54:43.461183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1133/2254] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:54:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:55:01.760487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:55:01.761919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:01.856253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b292c6f-9c9e-43d5-bd91-41ed6237bc3b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:55:01.862142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:55:01.863318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:01.863602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:01.863859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:01.865648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:01.877207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:55:01.879380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae25d8-5981-4e93-a394-4a01ff175c3c service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:55:01.889594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1122/2255] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:55:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:55:06.680001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:55:06.737772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581c2e6-b53a-4f48-a59b-401abd95cccf service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:55:06.742873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:55:06.743722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:06.744004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:06.744279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] PATH_INFO: `/v3/users/ee8a62866c2f4b838f1691c24366ca57` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:06.745478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee8a62866c2f4b838f1691c24366ca57)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:55:06.747381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:55:06.790539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee8a62866c2f4b838f1691c24366ca57'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:06.791175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee8a62866c2f4b838f1691c24366ca57'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:06.797734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee8a62866c2f4b838f1691c24366ca57'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:06.805705 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee8a62866c2f4b838f1691c24366ca57'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:06.806308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee8a62866c2f4b838f1691c24366ca57'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:06.823207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1134/2256] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:55:06 2022] DELETE /identity/v3/users/ee8a62866c2f4b838f1691c24366ca57 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:55:06.827754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:06.828234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:06.828346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:06.846843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:55:06.848061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:06.905723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1123/2257] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:55:06.915630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:55:06.917021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:06.971867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b28ea-0609-407e-ab5d-5b6c9921aae9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:55:06.975412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb99e66-0f7a-44b1-9bcc-89d23d800480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:55:06.976067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb99e66-0f7a-44b1-9bcc-89d23d800480 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:06.976343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb99e66-0f7a-44b1-9bcc-89d23d800480 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:06.976612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb99e66-0f7a-44b1-9bcc-89d23d800480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:06.985272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb99e66-0f7a-44b1-9bcc-89d23d800480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:55:06.986680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb99e66-0f7a-44b1-9bcc-89d23d800480 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:55:06.996324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1135/2258] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:55:07.256132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:55:07.257560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:07.269342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6c30-8c5b-4a88-a25b-f27506c39b46 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:55:07.274042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:55:07.274884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:07.275163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:07.275454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] PATH_INFO: `/v3/projects/21e82b4b2fb04a989adfdc1f168850f2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:07.276650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e82b4b2fb04a989adfdc1f168850f2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:55:07.278585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:55:07.325213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e82b4b2fb04a989adfdc1f168850f2', 'request_id': 'req-52097eec-8b96-420a-94f8-d0a1c8368a76'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:07.330311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e82b4b2fb04a989adfdc1f168850f2', 'request_id': 'req-52097eec-8b96-420a-94f8-d0a1c8368a76'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:07.336141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e82b4b2fb04a989adfdc1f168850f2', 'request_id': 'req-52097eec-8b96-420a-94f8-d0a1c8368a76'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:07.348958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] The token cache is being invalidate because project 21e82b4b2fb04a989adfdc1f168850f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:55:07.349238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:07.349847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-52097eec-8b96-420a-94f8-d0a1c8368a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:55:07.351357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1124/2259] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:55:07 2022] DELETE /identity/v3/projects/21e82b4b2fb04a989adfdc1f168850f2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:55:26.057354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:26.057841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:26.057945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:26.075022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] MFA Rules not processed for user `8fec0b9b99934e90b5c98eb43ca4a448`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:55:26.076210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:26.139156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1136/2260] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:55:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:55:49.306089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:49.306636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:49.306636 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:49.364775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:55:49.366003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:49.432676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1125/2261] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:55:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:55:49.472161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:55:49.473559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:55:49.558799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a31cc-5f22-4193-b263-916124f5affc None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:55:49.563709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:55:49.564338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:55:49.564614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:55:49.564917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:55:49.573190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:55:49.574745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:55:49.582461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1137/2262] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:55:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:56:01.686789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:01.688160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:01.740707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5032-f1d1-4da7-8c47-2b1a314d4b53 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:01.744151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:01.744805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:01.745082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:01.745350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:01.747044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:01.824456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:01.825908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac2ee0-5f31-43a4-8aa8-4cc4447f6998 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:01.845838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1126/2263] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:02.180921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:02.182358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:02.194315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ea63-da4e-45be-ba85-92b076c372fb service swift] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:02.197907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:02.198523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:02.198801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:02.199074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:02.200794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:02.213438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:02.214853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c35aab-8b8e-48e7-b4a5-27a1390e01b3 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:02.225433 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1138/2264] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:04.031960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2fc96-114c-4417-8745-c96f6701c5d6 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.032793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2fc96-114c-4417-8745-c96f6701c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.032793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2fc96-114c-4417-8745-c96f6701c5d6 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.034277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1127/2265] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:56:04.040298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10e7fa-5048-4ae9-92bc-196e458fc5d3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.040866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10e7fa-5048-4ae9-92bc-196e458fc5d3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.040866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10e7fa-5048-4ae9-92bc-196e458fc5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.078467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7a10e7fa-5048-4ae9-92bc-196e458fc5d3 None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:56:04.079605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7a10e7fa-5048-4ae9-92bc-196e458fc5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:04.089562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1139/2266] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:56:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:56:04.101561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2fc96-114c-4417-8745-c96f6701c5d6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:04.103260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cbd2fc96-114c-4417-8745-c96f6701c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:04.163441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2fc96-114c-4417-8745-c96f6701c5d6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:04.167662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c2d1-07cb-49ed-a2e1-3213bbadf97c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:04.168586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c2d1-07cb-49ed-a2e1-3213bbadf97c service glance] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.168828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c2d1-07cb-49ed-a2e1-3213bbadf97c service glance] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.169108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c2d1-07cb-49ed-a2e1-3213bbadf97c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.179549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2c2d1-07cb-49ed-a2e1-3213bbadf97c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:04.181021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2c2d1-07cb-49ed-a2e1-3213bbadf97c service glance] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:04.192307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1128/2267] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:04.262097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b0a4e-7d2e-49be-8519-e7a3c0fe596b None None] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.262413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b0a4e-7d2e-49be-8519-e7a3c0fe596b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.262690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b0a4e-7d2e-49be-8519-e7a3c0fe596b None None] PATH_INFO: `/` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.264219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1140/2268] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 16 18:56:04.276081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5c07-ee94-4aa5-ae5c-8bb37a92d439 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.276363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5c07-ee94-4aa5-ae5c-8bb37a92d439 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.276650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5c07-ee94-4aa5-ae5c-8bb37a92d439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.295302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-53df5c07-ee94-4aa5-ae5c-8bb37a92d439 None None] MFA Rules not processed for user `e653231605f345b0846ac42508e7d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:56:04.296497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-53df5c07-ee94-4aa5-ae5c-8bb37a92d439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:04.303095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1129/2269] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:56:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:56:04.314451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b0a4e-7d2e-49be-8519-e7a3c0fe596b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:04.325563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b0a4e-7d2e-49be-8519-e7a3c0fe596b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:04.329055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9e183-c951-4f47-b515-26e022fcfa9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:04.329621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9e183-c951-4f47-b515-26e022fcfa9a service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.329899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9e183-c951-4f47-b515-26e022fcfa9a service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.330170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9e183-c951-4f47-b515-26e022fcfa9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.337541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9e183-c951-4f47-b515-26e022fcfa9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:04.339006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9e183-c951-4f47-b515-26e022fcfa9a service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:04.346404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1141/2270] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:04.430370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f342-0151-402f-9042-105599cde1f1 None None] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.430690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f342-0151-402f-9042-105599cde1f1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.430968 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f342-0151-402f-9042-105599cde1f1 None None] PATH_INFO: `/` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.432339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1130/2271] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 16 18:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 16 18:56:04.437125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.437412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.437687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.455940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] MFA Rules not processed for user `b163f5034b7a4d96a224c07b9c1428c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:56:04.457170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:04.466142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1142/2272] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 16 18:56:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:56:04.476230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f342-0151-402f-9042-105599cde1f1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:04.487436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f342-0151-402f-9042-105599cde1f1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:04.490972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f520f-cdd3-4acb-ac33-4af38281bcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:04.491589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f520f-cdd3-4acb-ac33-4af38281bcfa service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.491873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f520f-cdd3-4acb-ac33-4af38281bcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.492146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f520f-cdd3-4acb-ac33-4af38281bcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.501898 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f520f-cdd3-4acb-ac33-4af38281bcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:04.503952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f520f-cdd3-4acb-ac33-4af38281bcfa service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:04.514271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1131/2273] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:04.576970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:04.578313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:04.641949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8090bc8-71d6-42bc-966a-7cc287bc29fc None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:04.646767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:04.647391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:04.647667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:04.647935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:04.649659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:04.660544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:04.662120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:04.671658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1143/2274] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:07.689524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:07.690036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:07.690223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:07.708762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:56:07.709990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:07.717363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1132/2275] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:56:07.730359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:07.741711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cc454-33c4-4f73-b911-23c926cbb298 service placement] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:07.746785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:07.747487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:07.747767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:07.748039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:07.757271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:07.759179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:07.770144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1144/2276] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:56:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:56:14.210270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:14.213669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:14.230096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204cd5c-9f09-402e-8d82-71b35d570a6a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:14.235635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:14.236470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:14.236813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:14.237091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:14.238902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:14.299309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:14.300942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:14.309653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1133/2277] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:21.564621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:21.573505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3d4d-8fef-460a-8a5b-e73ac67ce024 service swift] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:21.577057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:21.577676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:21.577957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:21.578233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:21.580562 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:21.590165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:21.591575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce346cc-3000-44d7-9c19-9647f397c81a service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:21.599391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1145/2278] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:21.630726 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:21.632201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:21.646067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cba61e-9759-4f06-b509-748e6be8acd8 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:21.651576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:21.652288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:21.652574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:21.652877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:21.654613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:21.715533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:21.717143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:21.725833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1134/2279] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:56:38.213456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fdf86-a09c-4753-b456-3b544265b74c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:38.215143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fdf86-a09c-4753-b456-3b544265b74c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:38.215143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fdf86-a09c-4753-b456-3b544265b74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:38.233946 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-012fdf86-a09c-4753-b456-3b544265b74c None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:56:38.235407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-012fdf86-a09c-4753-b456-3b544265b74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:38.243238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1146/2280] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 16 18:56:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:56:38.255686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:38.267274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701528c-decc-447e-b1f7-0b973b9f836a service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:38.272133 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:38.272796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:38.273074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:38.273344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:38.281040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:38.283266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:38.290994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1135/2281] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:56:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:56:49.507509 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:49.508059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:49.508059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:49.525134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] MFA Rules not processed for user `3b39f00235c34612b591cd2d974d9969`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:56:49.526313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:49.533301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1147/2282] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:56:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:56:55.582686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:56:55.597878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00387a-41d7-4a89-b84d-aa61744760ee service swift] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:56:55.603208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:56:55.603869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:56:55.604172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:56:55.604439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:56:55.606214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:56:55.618428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:56:55.619984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:56:55.630796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1136/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:08.712219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:08.724409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1cb3d-17b5-4a77-a9f8-dfd2d4e76abe None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:08.730083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:08.730711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:08.730986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:08.731256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:08.732985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:08.791445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:08.793051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:08.801655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1148/2284] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:26.392500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:26.403578 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9df52-672b-43c9-8dfd-9aaa00189084 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:26.409361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:26.410030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:26.410311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:26.410586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:26.412309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:26.423653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:26.425464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61797ce0-2dd2-4465-b122-bef88ef20536 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:26.438370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1137/2285] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:30.602921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:30.617361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b889f-aa28-4501-9025-89f1fa71a0a7 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:30.624430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:30.625378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:30.625913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:30.626391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:30.631138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:30.651550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:30.653788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:30.686787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1149/2286] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:36.744406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:36.744895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:36.745011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:36.763177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:36.764388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:36.771493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1138/2287] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 16 18:57:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:57:36.796540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:36.807393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1fcb-8494-4d2b-967d-befc699bf5a4 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:36.812870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150b5b9-6de1-45b6-8e52-87dbe5b30f91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:36.813496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b5b9-6de1-45b6-8e52-87dbe5b30f91 service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:36.813775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b5b9-6de1-45b6-8e52-87dbe5b30f91 service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:36.814056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150b5b9-6de1-45b6-8e52-87dbe5b30f91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:36.821882 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150b5b9-6de1-45b6-8e52-87dbe5b30f91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:36.823442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150b5b9-6de1-45b6-8e52-87dbe5b30f91 service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:36.831489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1150/2288] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 18:57:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 18:57:37.169306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:37.178987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c655c-a8f0-4f03-be62-3b883894bf5a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:37.182873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:37.183515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:37.183790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:37.184063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:37.185841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:37.247668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:37.249098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:37.259017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1139/2289] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:41.788235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:41.788854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:41.788854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:41.807250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] MFA Rules not processed for user `8fec0b9b99934e90b5c98eb43ca4a448`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:41.808438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:41.815988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1151/2290] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:57:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:57:42.271448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:42.282313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7ae80-f9bb-4cb8-8af5-d7cfd37d3ed2 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:42.286030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:42.286660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:42.286934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:42.287203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:42.288970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:42.302631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:42.304986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:42.313938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1140/2291] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:44.900443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:44.902131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:44.925643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95768d69-294c-4de5-82f2-b1fb030c07b3 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:44.930421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:44.931298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:44.937608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:44.938077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] PATH_INFO: `/v3/users/18a2a873aeae41de8b25c2095c645479` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:44.939352 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a2a873aeae41de8b25c2095c645479)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:44.941323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:45.007901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a2a873aeae41de8b25c2095c645479'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.008843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a2a873aeae41de8b25c2095c645479'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.018237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a2a873aeae41de8b25c2095c645479'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.027416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a2a873aeae41de8b25c2095c645479'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.028418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a2a873aeae41de8b25c2095c645479'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.053105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1152/2292] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:57:44 2022] DELETE /identity/v3/users/18a2a873aeae41de8b25c2095c645479 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:57:45.246534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:45.261556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13648c31-c7ea-4ccf-a911-0292a3331a54 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:45.348602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:45.349778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:45.350234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:45.350687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] PATH_INFO: `/v3/projects/0e91eab6ef81473c97f4c7dfd04506f5` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:45.352081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e91eab6ef81473c97f4c7dfd04506f5)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:45.354265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:45.407283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e91eab6ef81473c97f4c7dfd04506f5', 'request_id': 'req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.412714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e91eab6ef81473c97f4c7dfd04506f5', 'request_id': 'req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.418593 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e91eab6ef81473c97f4c7dfd04506f5', 'request_id': 'req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.430808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] The token cache is being invalidate because project 0e91eab6ef81473c97f4c7dfd04506f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:57:45.431096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.431932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.433571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1141/2293] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:57:45 2022] DELETE /identity/v3/projects/0e91eab6ef81473c97f4c7dfd04506f5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:57:45.438122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:45.439589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:45.453022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39daf99-096c-46fe-9743-5fdf9407d1f8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:45.510984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:45.511456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:45.511733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:45.511999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] PATH_INFO: `/v3/users/335e69de31144d358a137db17177a038` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:45.513795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335e69de31144d358a137db17177a038)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:45.515775 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:45.572891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335e69de31144d358a137db17177a038'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.573866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335e69de31144d358a137db17177a038'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.582383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335e69de31144d358a137db17177a038'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.597956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335e69de31144d358a137db17177a038'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.598727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d8ee12b7-6360-4942-8667-11c4210d0465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335e69de31144d358a137db17177a038'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:45.619143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1153/2294] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:57:45 2022] DELETE /identity/v3/users/335e69de31144d358a137db17177a038 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:57:45.862419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:45.863921 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:45.898566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b5bd-6cfa-4fbe-b401-62eb0fa2d2a1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:45.971732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:45.973336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:45.973627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:45.973901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] PATH_INFO: `/v3/projects/aec1c5cf63d74c6c8cb11dee348744b1` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:45.975157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec1c5cf63d74c6c8cb11dee348744b1)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:45.978495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.028064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec1c5cf63d74c6c8cb11dee348744b1', 'request_id': 'req-a1b49cca-3b48-4297-8576-4a676e88a18b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:46.034412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec1c5cf63d74c6c8cb11dee348744b1', 'request_id': 'req-a1b49cca-3b48-4297-8576-4a676e88a18b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:46.040530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec1c5cf63d74c6c8cb11dee348744b1', 'request_id': 'req-a1b49cca-3b48-4297-8576-4a676e88a18b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:46.053741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] The token cache is being invalidate because project aec1c5cf63d74c6c8cb11dee348744b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:57:46.054021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:46.054642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a1b49cca-3b48-4297-8576-4a676e88a18b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:57:46.056220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1142/2295] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:57:45 2022] DELETE /identity/v3/projects/aec1c5cf63d74c6c8cb11dee348744b1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:57:46.061480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.062016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.062137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.079988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:46.081249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:46.139778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1154/2296] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:57:46.147469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.147820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.148106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.167105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:46.168304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:46.176983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1143/2297] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:57:46.181002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.190293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60671aa-0354-46b5-be0b-a9999d3e6913 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.194826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.195471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.195751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.196023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.196782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:57:46.197087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.198668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.209836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1155/2298] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:57:46.214243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.223721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bb36-8259-4037-a38a-c9d06377e9c7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.228392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.229070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.229351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.229632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.230315 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.232332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.269891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1144/2299] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:57:46 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:57:46.274021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.284494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971e0c8-6672-41dc-baf8-6951ee22d3be admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.339865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.340584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.340948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.341251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.342549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.344506 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.363835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1156/2300] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:57:46 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:57:46.367914 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.377959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6838af-cd3c-4e35-b581-c6fec45a27a1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.382524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.383171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.383454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.383727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.384445 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.386088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.399468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1145/2301] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:57:46.400539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.411248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bac99-93c6-4f8f-99df-d2bc2583f53e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.416232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1f33-9d30-407d-8bfe-396ecc926770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.416932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1f33-9d30-407d-8bfe-396ecc926770 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.417217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1f33-9d30-407d-8bfe-396ecc926770 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.417499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1f33-9d30-407d-8bfe-396ecc926770 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.418311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c1f33-9d30-407d-8bfe-396ecc926770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.419871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c1f33-9d30-407d-8bfe-396ecc926770 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.436156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1157/2302] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:57:46.437389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.447827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c6715-a2a2-4fe3-a685-d5f7cd897ddf admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.453397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.454280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.454558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.455050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] PATH_INFO: `/v3/projects/3efbfa7562e44ff686595413de7e3057/users/e5666513102246cfa1f033069fa8ecbd/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.473783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efbfa7562e44ff686595413de7e3057, user_id=e5666513102246cfa1f033069fa8ecbd, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.476833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.491950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1146/2303] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:57:46 2022] PUT /identity/v3/projects/3efbfa7562e44ff686595413de7e3057/users/e5666513102246cfa1f033069fa8ecbd/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:57:46.496918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4ad64-ff34-4271-9cb7-8cafb2d10848 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.497237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4ad64-ff34-4271-9cb7-8cafb2d10848 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.497473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4ad64-ff34-4271-9cb7-8cafb2d10848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.516249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c9c4ad64-ff34-4271-9cb7-8cafb2d10848 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:46.517460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c9c4ad64-ff34-4271-9cb7-8cafb2d10848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:46.585244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1158/2304] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:57:46.594813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:46.596525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:46.651301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234613cc-6a9b-450a-a776-55242255c81d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:46.655433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:46.656082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:46.656367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:46.656956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:46.671793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:46.673303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:46.682803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1147/2305] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:51.372173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.373238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.373329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.407997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] MFA Rules not processed for user `e5666513102246cfa1f033069fa8ecbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:51.409487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:51.489256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1159/2306] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:57:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:57:51.490471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:51.505460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944f8f5-2552-4e3d-b70a-362d40c5fe71 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:51.513553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:51.514218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.514498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.514773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.517560 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:51.519574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:51.548676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1148/2307] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:57:51 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:57:51.552267 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:51.565473 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26774c8e-1e34-4a4b-acf1-d71c02854947 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:51.628587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:51.629365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.629741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.630113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.630892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:51.633108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:51.657976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1160/2308] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:57:51 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:57:51.661981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:51.674203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140460da-23a8-4927-b511-a0bdcd913253 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:51.678913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:51.679630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.679919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.680196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.680959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:51.682646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:51.695999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1149/2309] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:57:51.700125 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:51.710410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90126096-a41b-42c8-8028-4537b23ade7b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:51.715055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:51.715951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.716236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.716513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] PATH_INFO: `/v3/projects/25bd40642a5b499f810f9d32d44a4cf2/users/c14a0556f40d409bb53542a418ac28d8/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.733499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25bd40642a5b499f810f9d32d44a4cf2, user_id=c14a0556f40d409bb53542a418ac28d8, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:51.736344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:51.749982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1161/2310] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:57:51 2022] PUT /identity/v3/projects/25bd40642a5b499f810f9d32d44a4cf2/users/c14a0556f40d409bb53542a418ac28d8/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:57:51.754009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:51.765385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31e0b9-66e6-4aed-98e8-2743011a58c4 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:51.821691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:51.822379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.822689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.822981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.823700 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:51.825355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:51.834779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1150/2311] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:57:51.839056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:51.849278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fac68-900b-47f3-ad9d-e7034b5cc632 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:51.853878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949220f-5db3-47f3-aa19-ad11ee1b27bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:51.854745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949220f-5db3-47f3-aa19-ad11ee1b27bc admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:51.855028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949220f-5db3-47f3-aa19-ad11ee1b27bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:51.855311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949220f-5db3-47f3-aa19-ad11ee1b27bc admin admin] PATH_INFO: `/v3/projects/25bd40642a5b499f810f9d32d44a4cf2/users/c14a0556f40d409bb53542a418ac28d8/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:51.857799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949220f-5db3-47f3-aa19-ad11ee1b27bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=25bd40642a5b499f810f9d32d44a4cf2, user_id=c14a0556f40d409bb53542a418ac28d8, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:51.860405 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949220f-5db3-47f3-aa19-ad11ee1b27bc admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:51.870486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1162/2312] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:57:51 2022] PUT /identity/v3/projects/25bd40642a5b499f810f9d32d44a4cf2/users/c14a0556f40d409bb53542a418ac28d8/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:57:52.180395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:52.182581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:52.274680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448b857-e168-4261-811c-52ed0e6ce254 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:52.349079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778f425-8a42-43bd-9490-34929547131d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:52.349906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778f425-8a42-43bd-9490-34929547131d service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:52.350192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778f425-8a42-43bd-9490-34929547131d service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:52.350465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778f425-8a42-43bd-9490-34929547131d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:52.353242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b778f425-8a42-43bd-9490-34929547131d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:52.368843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778f425-8a42-43bd-9490-34929547131d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:52.372513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778f425-8a42-43bd-9490-34929547131d service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:52.386256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1151/2313] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:57:58.522453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a766c62-1be4-4103-beb2-1f5fd5668b23 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:58.523161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a766c62-1be4-4103-beb2-1f5fd5668b23 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:58.523475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a766c62-1be4-4103-beb2-1f5fd5668b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:58.594130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1a766c62-1be4-4103-beb2-1f5fd5668b23 None None] MFA Rules not processed for user `c14a0556f40d409bb53542a418ac28d8`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:57:58.595596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1a766c62-1be4-4103-beb2-1f5fd5668b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:57:58.788890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1163/2314] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:57:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:57:58.805136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778f425-8a42-43bd-9490-34929547131d service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:57:58.819391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778f425-8a42-43bd-9490-34929547131d service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:57:58.827970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:57:58.829047 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:57:58.829541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:57:58.830078 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:57:58.937622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:57:58.939239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:57:58.961373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1152/2315] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:57:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:00.518345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:00.518880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:00.518880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:00.542549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] MFA Rules not processed for user `867f26290c254777b65cb739263d708a`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:00.543820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:00.662616 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1164/2316] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:01.135182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:01.235690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e0ed6-a495-4a98-813e-39a35e8f39f7 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:01.240298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:01.245195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:01.245733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:01.246236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:01.248995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:01.261291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:01.264127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:01.277278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1153/2317] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:18.423205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:18.432834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76712a6b-2683-4ef0-a629-4c0c63fa7442 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:18.436935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:18.437573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:18.438491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:18.438491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:18.439899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:18.501124 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:18.502558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:18.512715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1165/2318] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 18:58:21.678785 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:21.680220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:21.694950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318351-fa41-4ec3-a404-c08fd2c97a48 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:21.700096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:21.700998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:21.701322 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:21.701599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] PATH_INFO: `/v3/users/3b39f00235c34612b591cd2d974d9969` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:21.702799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b39f00235c34612b591cd2d974d9969)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:21.704823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:21.752957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b39f00235c34612b591cd2d974d9969'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:21.759459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b39f00235c34612b591cd2d974d9969'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:21.760116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b39f00235c34612b591cd2d974d9969'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:21.768161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b39f00235c34612b591cd2d974d9969'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:21.768842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-eeb56721-ec21-438c-819a-cdccff16aefb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b39f00235c34612b591cd2d974d9969'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:21.786136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1154/2319] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:21 2022] DELETE /identity/v3/users/3b39f00235c34612b591cd2d974d9969 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:21.952537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:21.968011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71480-752f-4c85-b7f5-2a562dec23e0 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.023006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.023846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.024120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.024386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] PATH_INFO: `/v3/projects/c8c41c631a51459199afd424bd8de81c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.025564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c41c631a51459199afd424bd8de81c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.027482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.070258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c41c631a51459199afd424bd8de81c', 'request_id': 'req-0ec78c94-8443-48c0-afcf-9861b5d20b23'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:22.076485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c41c631a51459199afd424bd8de81c', 'request_id': 'req-0ec78c94-8443-48c0-afcf-9861b5d20b23'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:22.083528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c41c631a51459199afd424bd8de81c', 'request_id': 'req-0ec78c94-8443-48c0-afcf-9861b5d20b23'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:22.093863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] The token cache is being invalidate because project c8c41c631a51459199afd424bd8de81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:22.094143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:22.094750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0ec78c94-8443-48c0-afcf-9861b5d20b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:22.096241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1166/2320] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:21 2022] DELETE /identity/v3/projects/c8c41c631a51459199afd424bd8de81c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:22.101333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.101625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.101909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.119837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:22.121602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:22.179151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1155/2321] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:22.187130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.187416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.187695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.205976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:22.207237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:22.215257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1167/2322] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:22.218984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.228391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff6877-8563-44a5-84f9-b20d58a7ec6d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.232896 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.233535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.233811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.234082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.234813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:58:22.235141 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.236738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.248140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1156/2323] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:58:22.252211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.261239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e0d6b-dfe2-4f4d-a93e-b1a23f745f29 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.267221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.267864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.268141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.268427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.269148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.271028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.299796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1168/2324] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:22 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:22.301149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.311461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8e321-fc9e-4a91-a327-364679340f4b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.365187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.365842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.366120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.366397 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.367064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.368995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.388405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1157/2325] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:22 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:22.392544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.402523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277faa1-ba6c-404d-be4d-18e582b482af admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.407333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.407972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.408249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.408518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.409262 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.410838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.421605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1169/2326] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:22.425571 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.435646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f7b22-5a31-47ac-be1c-7d0023371c2b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.440208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c3ef5-c1f7-4186-91da-0ba1011e1555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.440890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c3ef5-c1f7-4186-91da-0ba1011e1555 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.441169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c3ef5-c1f7-4186-91da-0ba1011e1555 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.441443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c3ef5-c1f7-4186-91da-0ba1011e1555 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.442159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c3ef5-c1f7-4186-91da-0ba1011e1555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.443730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c3ef5-c1f7-4186-91da-0ba1011e1555 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.453113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1158/2327] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:22.456976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.466242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fbc48-824c-4808-b88a-158810272ed8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.470839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.471764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.471954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.472222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] PATH_INFO: `/v3/projects/4ea31a2167ba4c12a33972a9c0a09613/users/b5117e7cfbc6490b8429252021b94f40/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.486862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea31a2167ba4c12a33972a9c0a09613, user_id=b5117e7cfbc6490b8429252021b94f40, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.489507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.503345 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1170/2328] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:22 2022] PUT /identity/v3/projects/4ea31a2167ba4c12a33972a9c0a09613/users/b5117e7cfbc6490b8429252021b94f40/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:22.513631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30607-6182-48bf-8211-352aff34b485 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.513920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30607-6182-48bf-8211-352aff34b485 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.514198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30607-6182-48bf-8211-352aff34b485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.542064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0da30607-6182-48bf-8211-352aff34b485 None None] MFA Rules not processed for user `b5117e7cfbc6490b8429252021b94f40`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:22.543797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0da30607-6182-48bf-8211-352aff34b485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:22.604408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1159/2329] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:22.615972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.617658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:22.687035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac790d8b-86cb-4c99-9a7d-c588de59b322 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.692480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e688a28-247b-44dd-9ab8-2a51af395765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.693134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e688a28-247b-44dd-9ab8-2a51af395765 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.693411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e688a28-247b-44dd-9ab8-2a51af395765 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.693679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e688a28-247b-44dd-9ab8-2a51af395765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.701640 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e688a28-247b-44dd-9ab8-2a51af395765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.703192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e688a28-247b-44dd-9ab8-2a51af395765 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:22.711652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1171/2330] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:22.906895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30607-6182-48bf-8211-352aff34b485 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:22.968194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30607-6182-48bf-8211-352aff34b485 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:22.972553 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:22.973440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:22.973717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:22.973990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] PATH_INFO: `/v3/users/b5117e7cfbc6490b8429252021b94f40` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:22.975156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5117e7cfbc6490b8429252021b94f40)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:22.977082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.020161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5117e7cfbc6490b8429252021b94f40'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.026872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5117e7cfbc6490b8429252021b94f40'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.027671 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5117e7cfbc6490b8429252021b94f40'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.035659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5117e7cfbc6490b8429252021b94f40'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.036318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5117e7cfbc6490b8429252021b94f40'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.052520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1160/2331] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:22 2022] DELETE /identity/v3/users/b5117e7cfbc6490b8429252021b94f40 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:23.057045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.057508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.057601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.074291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:23.075451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:23.131228 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1172/2332] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:23.140988 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.142427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:23.197539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a989854-8e4a-4c99-810b-128a1bc01bed admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.201127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb57d3d-a731-454e-ab2e-125ceda0a298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.201752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57d3d-a731-454e-ab2e-125ceda0a298 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.202028 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57d3d-a731-454e-ab2e-125ceda0a298 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.202295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57d3d-a731-454e-ab2e-125ceda0a298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.211153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb57d3d-a731-454e-ab2e-125ceda0a298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.212556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb57d3d-a731-454e-ab2e-125ceda0a298 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.222006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1161/2333] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:23.482013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.483854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:23.495767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f1ed8-2c13-4a9e-8436-79206b6a8526 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.500369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.501324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.501670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.502013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] PATH_INFO: `/v3/projects/4ea31a2167ba4c12a33972a9c0a09613` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.503265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea31a2167ba4c12a33972a9c0a09613)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.505546 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.552412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea31a2167ba4c12a33972a9c0a09613', 'request_id': 'req-026361f8-be89-4911-9e32-8957aa4b259d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.557699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea31a2167ba4c12a33972a9c0a09613', 'request_id': 'req-026361f8-be89-4911-9e32-8957aa4b259d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.564663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea31a2167ba4c12a33972a9c0a09613', 'request_id': 'req-026361f8-be89-4911-9e32-8957aa4b259d'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.575411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] The token cache is being invalidate because project 4ea31a2167ba4c12a33972a9c0a09613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:23.575684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.576318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-026361f8-be89-4911-9e32-8957aa4b259d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:23.577895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1173/2334] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:23 2022] DELETE /identity/v3/projects/4ea31a2167ba4c12a33972a9c0a09613 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:23.582765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.583053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.583329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.601142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:23.602356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:23.659972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1162/2335] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:23.668216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.668514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.668842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.686240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:23.687400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:23.695158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1174/2336] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:23.699306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.708537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e11f6-f772-4b07-8af0-00dc8f69587b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.713018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.713656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.713932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.714199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.715442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:58:23.715739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.717349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.728283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1163/2337] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:58:23.732917 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.741661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc01b38-b757-4db6-a901-f27049e095ba None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.746184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.746808 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.747084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.747351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.748023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.749912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.774069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1175/2338] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:23 2022] POST /identity/v3/projects => generated 355 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:58:23.778596 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.789072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c7ff4-88ea-4caa-8a69-c5441484d069 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.848287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.849385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.849789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.850096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.850936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.852866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.872272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1164/2339] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:23 2022] POST /identity/v3/users => generated 278 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:23.876833 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.887569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d4390-c2a5-4d57-af00-341237653fe3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.892117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.892803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.893081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.893350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.894057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.895626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.906487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1176/2340] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:23.910411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.920526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3c21-2bd8-4937-a346-ca65b7b03ccd admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.926013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eb3ef-6dfb-4ef9-be04-6e85773b287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.926668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eb3ef-6dfb-4ef9-be04-6e85773b287f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.926951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eb3ef-6dfb-4ef9-be04-6e85773b287f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.927227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eb3ef-6dfb-4ef9-be04-6e85773b287f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.927947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eb3ef-6dfb-4ef9-be04-6e85773b287f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.929572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eb3ef-6dfb-4ef9-be04-6e85773b287f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.939258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1165/2341] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:58:23.943282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:23.953419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67286ed9-3a79-41f9-a69a-b08a79d8a7cd admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:23.958547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:23.959540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:23.959845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:23.960115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] PATH_INFO: `/v3/projects/e4560591bc9c42a9a2cbfcae486311a9/users/e1e442e28a55441b83eb0c252a197246/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:23.976082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4560591bc9c42a9a2cbfcae486311a9, user_id=e1e442e28a55441b83eb0c252a197246, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:23.979004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:23.991245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1177/2342] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:23 2022] PUT /identity/v3/projects/e4560591bc9c42a9a2cbfcae486311a9/users/e1e442e28a55441b83eb0c252a197246/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:58:24.001235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.001534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.001819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.029404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] MFA Rules not processed for user `e1e442e28a55441b83eb0c252a197246`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:24.030616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:24.090113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1166/2343] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:24.094492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.154240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003e3f3-5423-4415-8908-959f460a6c5d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.160807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.161573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.161857 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.162134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.162840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.165019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.189258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1178/2344] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:24 2022] POST /identity/v3/projects => generated 357 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:24.193484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.204301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f75cf8-fa07-4812-b246-2ed3ef922bd4 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.257007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.257599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.257876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.258147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.258823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.260765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.279742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1167/2345] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:24 2022] POST /identity/v3/users => generated 285 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:24.284076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.294129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6309fb-14ef-48bd-840c-3320e337b1ff admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.298725 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.299365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.299639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.299908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.300612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.302201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.312763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1179/2346] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:58:24.316649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.327555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e95b8-a91d-44eb-804e-905bf3a4eaec admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.332468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.333361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.333665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.333938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] PATH_INFO: `/v3/projects/d373b65c2e1a4e67a99ebeb403d93e3c/users/c3f0b89f661f4cd9ac2a426592024cf5/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.348834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d373b65c2e1a4e67a99ebeb403d93e3c, user_id=c3f0b89f661f4cd9ac2a426592024cf5, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.351461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.366770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1168/2347] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:24 2022] PUT /identity/v3/projects/d373b65c2e1a4e67a99ebeb403d93e3c/users/c3f0b89f661f4cd9ac2a426592024cf5/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:24.371146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.380821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b473f91-5675-4965-ab99-4435fe8f6052 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.438240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6219c-05b1-4a47-b910-e078b3d3b129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.439167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6219c-05b1-4a47-b910-e078b3d3b129 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.439627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6219c-05b1-4a47-b910-e078b3d3b129 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.439742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6219c-05b1-4a47-b910-e078b3d3b129 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.440420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6219c-05b1-4a47-b910-e078b3d3b129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.442032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6219c-05b1-4a47-b910-e078b3d3b129 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.452203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1180/2348] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:24.458045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.468668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fe2d8-cf4a-41fb-94b7-d66a7f87e2f4 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.468997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.469283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.469560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.473855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edea00-ffc5-48fa-b6bf-89003f57565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.474765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edea00-ffc5-48fa-b6bf-89003f57565b admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.475048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edea00-ffc5-48fa-b6bf-89003f57565b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.475326 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edea00-ffc5-48fa-b6bf-89003f57565b admin admin] PATH_INFO: `/v3/projects/d373b65c2e1a4e67a99ebeb403d93e3c/users/c3f0b89f661f4cd9ac2a426592024cf5/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.477368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edea00-ffc5-48fa-b6bf-89003f57565b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d373b65c2e1a4e67a99ebeb403d93e3c, user_id=c3f0b89f661f4cd9ac2a426592024cf5, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.480056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edea00-ffc5-48fa-b6bf-89003f57565b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.486907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] MFA Rules not processed for user `e5666513102246cfa1f033069fa8ecbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:24.488095 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:24.490514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1169/2349] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:24 2022] PUT /identity/v3/projects/d373b65c2e1a4e67a99ebeb403d93e3c/users/c3f0b89f661f4cd9ac2a426592024cf5/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:24.502533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.502819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.503095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.529231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] MFA Rules not processed for user `c3f0b89f661f4cd9ac2a426592024cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:24.530572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:24.547994 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1181/2350] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:24.596884 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1170/2351] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:24 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:24.608493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.609911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:24.675615 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05bbe8-ea11-4ed8-b92d-8b361463e387 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.679284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91ec12-1530-4aac-9505-7efbc30aea57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.679919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ec12-1530-4aac-9505-7efbc30aea57 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.680200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ec12-1530-4aac-9505-7efbc30aea57 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.680474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ec12-1530-4aac-9505-7efbc30aea57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.690168 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91ec12-1530-4aac-9505-7efbc30aea57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.691598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91ec12-1530-4aac-9505-7efbc30aea57 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.701925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1182/2352] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:24 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:24.774300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:24.784654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c118fa5-978a-4956-b923-e2fa31959c8b None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:24.788348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:24.789048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:24.789362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:24.789595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:24.853338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:24.854843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:24.865200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1171/2353] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:24 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:33.850420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2544d4-3064-4c05-a844-a559c233935c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:33.850979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2544d4-3064-4c05-a844-a559c233935c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:33.850979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2544d4-3064-4c05-a844-a559c233935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:33.868945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-dc2544d4-3064-4c05-a844-a559c233935c None None] MFA Rules not processed for user `e5c547a57e5e44fb8a1c89558b480991`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:33.871199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-dc2544d4-3064-4c05-a844-a559c233935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:33.940114 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1183/2354] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:39.660985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:39.722165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1984e5c-c62f-4bdb-a88f-0bc046d8c81e service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:39.726994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:39.727784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:39.728062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:39.728367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] PATH_INFO: `/v3/users/e1e442e28a55441b83eb0c252a197246` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:39.730091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e442e28a55441b83eb0c252a197246)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:39.731984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:39.775523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e442e28a55441b83eb0c252a197246'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:39.782453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e442e28a55441b83eb0c252a197246'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:39.783106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e442e28a55441b83eb0c252a197246'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:39.790965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e442e28a55441b83eb0c252a197246'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:39.792335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e442e28a55441b83eb0c252a197246'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:39.809793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1172/2355] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:39 2022] DELETE /identity/v3/users/e1e442e28a55441b83eb0c252a197246 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:39.814396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:39.814884 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:39.815096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:39.833111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:39.834410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:39.890794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1184/2356] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:39.900304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:39.901736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:39.956833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43b59f-0d0d-439f-95ad-1b221e50fb40 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:39.960737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:39.961405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:39.961680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:39.961947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:39.974838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:39.976277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:39.988051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1173/2357] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:40.243697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:40.245470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:40.258968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b3145-dffa-4e06-9095-9b84a0963209 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:40.263820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:40.265231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:40.265693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:40.265978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] PATH_INFO: `/v3/projects/e4560591bc9c42a9a2cbfcae486311a9` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:40.267172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4560591bc9c42a9a2cbfcae486311a9)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:40.269227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:40.317401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4560591bc9c42a9a2cbfcae486311a9', 'request_id': 'req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.323641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4560591bc9c42a9a2cbfcae486311a9', 'request_id': 'req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.331977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4560591bc9c42a9a2cbfcae486311a9', 'request_id': 'req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.346089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] The token cache is being invalidate because project e4560591bc9c42a9a2cbfcae486311a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:40.346596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.347644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.349945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1185/2358] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:40 2022] DELETE /identity/v3/projects/e4560591bc9c42a9a2cbfcae486311a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:40.353850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:40.363904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f4ce-4d34-42b5-bb0c-a2a2793d75d2 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:40.418892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:40.419761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:40.420041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:40.420312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] PATH_INFO: `/v3/users/c3f0b89f661f4cd9ac2a426592024cf5` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:40.421526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f0b89f661f4cd9ac2a426592024cf5)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:40.423437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:40.467966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f0b89f661f4cd9ac2a426592024cf5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.474983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f0b89f661f4cd9ac2a426592024cf5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.475766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f0b89f661f4cd9ac2a426592024cf5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.485551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f0b89f661f4cd9ac2a426592024cf5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.486437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-ae8e229e-50a8-401e-ad10-aac1395af04e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f0b89f661f4cd9ac2a426592024cf5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.505852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1174/2359] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:40 2022] DELETE /identity/v3/users/c3f0b89f661f4cd9ac2a426592024cf5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:40.800263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:40.801691 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:40.817839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21f69d-5ead-4c62-8a9f-b90d3281f4ba admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:40.895988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:40.897330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:40.897645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:40.898397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] PATH_INFO: `/v3/projects/d373b65c2e1a4e67a99ebeb403d93e3c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:40.901015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d373b65c2e1a4e67a99ebeb403d93e3c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:40.903113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:40.967361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd373b65c2e1a4e67a99ebeb403d93e3c', 'request_id': 'req-d4f7162b-daae-4e20-93db-a64c2b9aff2a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.973374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd373b65c2e1a4e67a99ebeb403d93e3c', 'request_id': 'req-d4f7162b-daae-4e20-93db-a64c2b9aff2a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.981360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd373b65c2e1a4e67a99ebeb403d93e3c', 'request_id': 'req-d4f7162b-daae-4e20-93db-a64c2b9aff2a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.994698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] The token cache is being invalidate because project d373b65c2e1a4e67a99ebeb403d93e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:40.995915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.996922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d4f7162b-daae-4e20-93db-a64c2b9aff2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:40.999639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1186/2360] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:40 2022] DELETE /identity/v3/projects/d373b65c2e1a4e67a99ebeb403d93e3c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:41.004274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.004792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.005262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.024940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:41.026471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:41.092678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1175/2361] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:41.100557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.100873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.101152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.119839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:41.121622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:41.129499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1187/2362] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:41.133751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.143886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a4113-cb97-4346-88d0-f647ba00c1d8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.148525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.149270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.149563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.149831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.150551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:58:41.150858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.152477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.163990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1176/2363] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:58:41.169427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.178477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f587b3f-4e47-4a3a-9ccc-a16e173a7ce1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.183699 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.184334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.184613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.184922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.185721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.187641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.213483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1188/2364] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:58:41.217517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.227056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695be635-c94c-4b78-bbee-f2aea331f4b7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.281466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.282036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.282312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.282580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.283241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.285170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.303772 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1177/2365] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/users => generated 292 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:41.307978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.317849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3a637-6b0f-4d20-ae02-60b657a68c75 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.322384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.323527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.323806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.324078 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.324825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.326393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.336333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1189/2366] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:41.340749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.350189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc391512-b200-4aee-8418-5a949d43a169 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.355008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96caa-7524-480a-bef1-4c0dfb1c1ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.355637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96caa-7524-480a-bef1-4c0dfb1c1ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.355916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96caa-7524-480a-bef1-4c0dfb1c1ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.356186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96caa-7524-480a-bef1-4c0dfb1c1ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.356926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96caa-7524-480a-bef1-4c0dfb1c1ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.358503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96caa-7524-480a-bef1-4c0dfb1c1ef3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.367964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1178/2367] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:41.372000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.381580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502aa15-9197-428c-a982-007c69445d92 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.386612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.387462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.387736 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.388005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] PATH_INFO: `/v3/projects/6c6f72d1dc4e41b09e261fe7f97ada8f/users/a3f804876ec8470ebf6a20c0d12ccc8b/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.402487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6f72d1dc4e41b09e261fe7f97ada8f, user_id=a3f804876ec8470ebf6a20c0d12ccc8b, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.405118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.417714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1190/2368] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:41 2022] PUT /identity/v3/projects/6c6f72d1dc4e41b09e261fe7f97ada8f/users/a3f804876ec8470ebf6a20c0d12ccc8b/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:41.428270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.428562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.428893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.455702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] MFA Rules not processed for user `a3f804876ec8470ebf6a20c0d12ccc8b`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:41.456939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:41.524329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1179/2369] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:41.528858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.588057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2cd4d-aa45-405b-969c-353197a78b52 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.593583 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.594227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.594501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.594770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.595452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.597362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.621604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1191/2370] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:41.625525 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.635857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18fed5-c167-4698-89b3-930f248390b3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.689327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.689983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.690264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.690539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.691220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.693166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.712903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1180/2371] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/users => generated 297 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:41.717326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.727153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600715-41f5-473b-8ada-b4c2308e6db3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.732308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.732986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.733264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.733536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.734250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.736256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.746393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1192/2372] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:41.750235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.759792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435a031-2619-441f-8718-9510e66fb41f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.764364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.765262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.765541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.765812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] PATH_INFO: `/v3/projects/e931a8feb9634bf88a5e98db2b42364a/users/836344bb0bec465a9e0f0aac9f8337e7/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.780241 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e931a8feb9634bf88a5e98db2b42364a, user_id=836344bb0bec465a9e0f0aac9f8337e7, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.783153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.796400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1181/2373] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:41 2022] PUT /identity/v3/projects/e931a8feb9634bf88a5e98db2b42364a/users/836344bb0bec465a9e0f0aac9f8337e7/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:41.800182 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.809995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45035436-7653-461e-9b63-01421a9705d2 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.864555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf342023-4392-499c-8b1d-710eb4cbd729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.865225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342023-4392-499c-8b1d-710eb4cbd729 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.865465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342023-4392-499c-8b1d-710eb4cbd729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.865741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342023-4392-499c-8b1d-710eb4cbd729 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.866453 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf342023-4392-499c-8b1d-710eb4cbd729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.868039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf342023-4392-499c-8b1d-710eb4cbd729 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.877487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1193/2374] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:58:41.881597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:41.892060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aed4c-cf3e-4588-9ec1-ed468a1c5465 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:41.896805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:41.897674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.897953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.898681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] PATH_INFO: `/v3/projects/e931a8feb9634bf88a5e98db2b42364a/users/836344bb0bec465a9e0f0aac9f8337e7/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.900688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e931a8feb9634bf88a5e98db2b42364a, user_id=836344bb0bec465a9e0f0aac9f8337e7, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:41.904009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:41.914389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1182/2375] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:41 2022] PUT /identity/v3/projects/e931a8feb9634bf88a5e98db2b42364a/users/836344bb0bec465a9e0f0aac9f8337e7/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:58:41.925164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:41.925448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:41.925720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:41.951293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] MFA Rules not processed for user `836344bb0bec465a9e0f0aac9f8337e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:41.952468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:42.016944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1194/2376] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:42.028103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:42.029915 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:42.086567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1001-37ed-4512-b30f-e13c214aa554 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:42.090810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:42.091460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:42.091729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:42.091998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:42.101116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:42.102540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:42.112167 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1183/2377] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:42.213169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:42.222931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241a4a9-46c5-45bf-9b80-78de6e20571b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:42.226467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:42.227089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:42.227364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:42.227630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:42.281190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:42.282596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:42.291072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1195/2378] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:43.547285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:43.548696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:43.603930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00e06-a9a3-40bf-bf31-a7f7a48f84ca service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:43.608067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:43.608759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:43.609038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:43.609307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:43.611128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:43.673680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:43.675201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325140d-6147-4834-96a7-e2d377c7c987 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:43.685391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1184/2379] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:44.849166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:44.922907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd030f-546e-4be4-98d4-bcef1c5a4bd2 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:44.927390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:44.928252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:44.928526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:44.928827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] PATH_INFO: `/v3/users/a3f804876ec8470ebf6a20c0d12ccc8b` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:44.929993 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f804876ec8470ebf6a20c0d12ccc8b)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:44.931889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:44.988066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f804876ec8470ebf6a20c0d12ccc8b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:44.988847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f804876ec8470ebf6a20c0d12ccc8b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:44.996548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f804876ec8470ebf6a20c0d12ccc8b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.005036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f804876ec8470ebf6a20c0d12ccc8b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.005863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f804876ec8470ebf6a20c0d12ccc8b'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.027896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1196/2380] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:44 2022] DELETE /identity/v3/users/a3f804876ec8470ebf6a20c0d12ccc8b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:45.032987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:45.033848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:45.034128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:45.052702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:45.053919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:45.113284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1185/2381] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:45.122770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:45.124277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:45.183310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0604f47-2cb3-4f1c-9877-5bf670954afc admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:45.187318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:45.187965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:45.188252 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:45.188525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:45.198369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:45.199880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:45.209254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1197/2382] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:45.497650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:45.499054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:45.510856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda289f-5e0f-4ed9-90aa-df739fd5cd49 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:45.515993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:45.516891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:45.517379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:45.517931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] PATH_INFO: `/v3/projects/6c6f72d1dc4e41b09e261fe7f97ada8f` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:45.521436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6f72d1dc4e41b09e261fe7f97ada8f)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:45.523424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:45.569719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6f72d1dc4e41b09e261fe7f97ada8f', 'request_id': 'req-864b24ae-d204-45ca-85fc-05772ce72b51'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.575805 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6f72d1dc4e41b09e261fe7f97ada8f', 'request_id': 'req-864b24ae-d204-45ca-85fc-05772ce72b51'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.581963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6f72d1dc4e41b09e261fe7f97ada8f', 'request_id': 'req-864b24ae-d204-45ca-85fc-05772ce72b51'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.594789 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] The token cache is being invalidate because project 6c6f72d1dc4e41b09e261fe7f97ada8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:45.595177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.595813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.597374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1186/2383] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:45 2022] DELETE /identity/v3/projects/6c6f72d1dc4e41b09e261fe7f97ada8f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:45.601275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:45.610757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a875a-9852-468d-bed8-1870be48aab7 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:45.665276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:45.666242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:45.666816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:45.666947 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] PATH_INFO: `/v3/users/836344bb0bec465a9e0f0aac9f8337e7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:45.668170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836344bb0bec465a9e0f0aac9f8337e7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:45.670360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:45.715419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836344bb0bec465a9e0f0aac9f8337e7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.716074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836344bb0bec465a9e0f0aac9f8337e7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.722686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836344bb0bec465a9e0f0aac9f8337e7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.731111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836344bb0bec465a9e0f0aac9f8337e7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.731730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e4766003-a82a-4e4d-bf9b-a276b9b4eea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836344bb0bec465a9e0f0aac9f8337e7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:45.748889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1198/2384] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:45 2022] DELETE /identity/v3/users/836344bb0bec465a9e0f0aac9f8337e7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:46.032109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.033795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:46.047826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b24ae-d204-45ca-85fc-05772ce72b51 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.110043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.110908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.111392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.111819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] PATH_INFO: `/v3/projects/e931a8feb9634bf88a5e98db2b42364a` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.113339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e931a8feb9634bf88a5e98db2b42364a)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.115249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.164358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e931a8feb9634bf88a5e98db2b42364a', 'request_id': 'req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:46.170456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e931a8feb9634bf88a5e98db2b42364a', 'request_id': 'req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:46.176791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e931a8feb9634bf88a5e98db2b42364a', 'request_id': 'req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:46.190428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] The token cache is being invalidate because project e931a8feb9634bf88a5e98db2b42364a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:46.191292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:46.191515 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fbea42ba-cb00-4d66-8358-9dbaf9a64b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:46.193964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1187/2385] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:46 2022] DELETE /identity/v3/projects/e931a8feb9634bf88a5e98db2b42364a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:46.199603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.201176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.201482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.219528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:46.220774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:46.284300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1199/2386] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:46.292323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.292612 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.292920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.312910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:46.314527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:46.325314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1188/2387] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:46.330207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.340740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc472e-73b2-40a6-aa03-8f9058ea4970 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.345902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.347187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.347187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.347187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.348019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:58:46.348391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.350150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.373919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1200/2388] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:58:46.376295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.391614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4a829-0d7e-4d59-b70b-b819573c5132 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.399465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.400155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.400432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.400752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.402192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.404364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.441465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1189/2389] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:46.446091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.456091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0442-2a33-412b-8183-590265ac72df admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.512137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.512828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.513107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.513378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.514066 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.515957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.537646 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1201/2390] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:58:46.542579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.553518 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef4cad-25fd-42fc-a7ff-a819c186d8f5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.558351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.559030 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.559513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.559795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.560536 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.562134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.573458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1190/2391] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 18:58:46.577327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.587034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042c0d4-f451-49ed-8bf7-64258af0f113 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.592251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c1031-b5a1-41dc-b982-7e473ea41abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.592924 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c1031-b5a1-41dc-b982-7e473ea41abb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.593207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c1031-b5a1-41dc-b982-7e473ea41abb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.593483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c1031-b5a1-41dc-b982-7e473ea41abb admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.594226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c1031-b5a1-41dc-b982-7e473ea41abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.595800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c1031-b5a1-41dc-b982-7e473ea41abb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.605395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1202/2392] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:46.609461 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.619386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a0aeb-41e2-4ba6-bdc9-c141d46d11c0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.624019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.624906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.625187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.625460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] PATH_INFO: `/v3/projects/5ca0bb173adb4c04be1f0505b223cedc/users/0597938151aa48ada95c76888132fbb5/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.639657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca0bb173adb4c04be1f0505b223cedc, user_id=0597938151aa48ada95c76888132fbb5, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.642358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.655376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1191/2393] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:46 2022] PUT /identity/v3/projects/5ca0bb173adb4c04be1f0505b223cedc/users/0597938151aa48ada95c76888132fbb5/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:46.666584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.666876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.667154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.692763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] MFA Rules not processed for user `0597938151aa48ada95c76888132fbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:46.693956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:46.763581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1203/2394] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:46.768089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.837031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a20a53-4870-439c-99e7-f566947a5415 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.843057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.843717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.844011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.844331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.844999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.846922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.874154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1192/2395] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:46.878176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.889855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821ec70-f5fe-47ac-97e4-91f21b822223 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.950754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.951458 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.951953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.952234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.953005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.959413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:46.978944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1204/2396] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:46 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:58:46.979971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:46.990564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c3175-538f-4d0e-ae8a-99f2ba8a4dd5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:46.995714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:46.996366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:46.996675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:46.996959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:46.997676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:46.999324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.009920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1193/2397] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:47.014326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.024000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a3dd7-d1b4-4771-9808-72873622cc21 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.029096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdbc04-9df4-4076-a3ef-d1625b99219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.029746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdbc04-9df4-4076-a3ef-d1625b99219d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.030028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdbc04-9df4-4076-a3ef-d1625b99219d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.030311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdbc04-9df4-4076-a3ef-d1625b99219d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.031277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdbc04-9df4-4076-a3ef-d1625b99219d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.032886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdbc04-9df4-4076-a3ef-d1625b99219d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.042713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1205/2398] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:47.047329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.063836 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8080f39-608d-4b1e-b5b1-8ae87b648110 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.069224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.070091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.070371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.070646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] PATH_INFO: `/v3/projects/f382a0c12659435c894d537dbc9c60eb/users/d8c01ac11d08478eb282931fd2062459/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.087999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f382a0c12659435c894d537dbc9c60eb, user_id=d8c01ac11d08478eb282931fd2062459, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.090813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.104418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1194/2399] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:47 2022] PUT /identity/v3/projects/f382a0c12659435c894d537dbc9c60eb/users/d8c01ac11d08478eb282931fd2062459/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:58:47.114599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.114961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.115245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.142901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] MFA Rules not processed for user `d8c01ac11d08478eb282931fd2062459`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:47.144540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:47.209964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1206/2400] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:47.214440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.275277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef653e-0c8a-4db6-8d7c-8532028c61af admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.280559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.281539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.281820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.282097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.282796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.284727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.308940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1195/2401] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:47 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:47.312753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.323519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea82c9-5ee0-4b61-8b6f-1146f6c6c78d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.376340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.377034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.377313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.377581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.378259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.380164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.405730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1207/2402] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:47 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:47.410122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.420164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169391d-795b-47da-afd3-e6466b10d908 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.424760 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.425410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.425691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.425960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.426790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.428396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.439187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1196/2403] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:47.443408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.453814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f31f7-4355-45bb-8f84-368165c767c9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.458515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.459407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.459690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.460019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] PATH_INFO: `/v3/projects/fd9e881bf76a4f4a881d924506c361f2/users/1ef1528105d14e0894a112fd91cc2772/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.475317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9e881bf76a4f4a881d924506c361f2, user_id=1ef1528105d14e0894a112fd91cc2772, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.477999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.490723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1208/2404] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:47 2022] PUT /identity/v3/projects/fd9e881bf76a4f4a881d924506c361f2/users/1ef1528105d14e0894a112fd91cc2772/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:47.495355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.505179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3300fe-15c5-405a-abc5-834807c63291 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.559972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bb4f2-631e-4063-a01f-f8a1007f3a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.560708 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bb4f2-631e-4063-a01f-f8a1007f3a1f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.560936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bb4f2-631e-4063-a01f-f8a1007f3a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.561207 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bb4f2-631e-4063-a01f-f8a1007f3a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.561920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bb4f2-631e-4063-a01f-f8a1007f3a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.563530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bb4f2-631e-4063-a01f-f8a1007f3a1f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.573281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1197/2405] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:47.577304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.587697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fe8cd-7f57-4e5f-9636-94bbdd6c489e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.592898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.593777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.594058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.594334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] PATH_INFO: `/v3/projects/fd9e881bf76a4f4a881d924506c361f2/users/1ef1528105d14e0894a112fd91cc2772/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.596295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9e881bf76a4f4a881d924506c361f2, user_id=1ef1528105d14e0894a112fd91cc2772, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.598936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.609650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1209/2406] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:47 2022] PUT /identity/v3/projects/fd9e881bf76a4f4a881d924506c361f2/users/1ef1528105d14e0894a112fd91cc2772/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:47.619284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.620074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.620357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.646502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] MFA Rules not processed for user `1ef1528105d14e0894a112fd91cc2772`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:47.647699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:47.714601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1198/2407] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:47.726024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.727703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:47.788294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf8c44-2eff-497d-9b99-98ebbeb0d685 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.792025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.792698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.792981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.793255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.802200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.803606 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.813443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1210/2408] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:47.855488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:47.857012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:47.868780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0bedc-764d-4b43-ad42-7ab26a47ae99 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:47.873111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64024497-c7c0-4cee-996f-7c105a58b6e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:47.873820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024497-c7c0-4cee-996f-7c105a58b6e6 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:47.874104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024497-c7c0-4cee-996f-7c105a58b6e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:47.874379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024497-c7c0-4cee-996f-7c105a58b6e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:47.929834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64024497-c7c0-4cee-996f-7c105a58b6e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:47.931316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64024497-c7c0-4cee-996f-7c105a58b6e6 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:47.940640 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1199/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:50.827298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:50.889636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a4153-c4ab-4d52-8fcc-0e530afa8794 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:50.895676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:50.896543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:50.896854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:50.897155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] PATH_INFO: `/v3/users/0597938151aa48ada95c76888132fbb5` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:50.898308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0597938151aa48ada95c76888132fbb5)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:50.900210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:50.947496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0597938151aa48ada95c76888132fbb5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:50.948155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0597938151aa48ada95c76888132fbb5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:50.954827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0597938151aa48ada95c76888132fbb5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:50.970837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0597938151aa48ada95c76888132fbb5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:50.971397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0597938151aa48ada95c76888132fbb5'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:50.989266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1211/2410] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:50 2022] DELETE /identity/v3/users/0597938151aa48ada95c76888132fbb5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:50.993753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:50.994271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:50.994466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:51.011729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:51.012922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:51.069095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1200/2411] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:51.078610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:51.079968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:51.134743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75021e-5a93-4c32-a46a-3d49181b21a1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:51.138875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:51.139501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:51.139779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:51.140047 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:51.148830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:51.150241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:51.160212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1212/2412] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:51.658372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:51.659959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:51.672164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb19f8e-d878-4f4f-9f0a-52d703fe0827 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:51.676699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:51.677556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:51.677835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:51.678104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] PATH_INFO: `/v3/projects/5ca0bb173adb4c04be1f0505b223cedc` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:51.679262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca0bb173adb4c04be1f0505b223cedc)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:51.681184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:51.726380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca0bb173adb4c04be1f0505b223cedc', 'request_id': 'req-c4281b4f-8cbd-4239-bf28-7a23c289588a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.731791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca0bb173adb4c04be1f0505b223cedc', 'request_id': 'req-c4281b4f-8cbd-4239-bf28-7a23c289588a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.737681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca0bb173adb4c04be1f0505b223cedc', 'request_id': 'req-c4281b4f-8cbd-4239-bf28-7a23c289588a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.749489 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] The token cache is being invalidate because project 5ca0bb173adb4c04be1f0505b223cedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:51.749773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.750380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.751935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1201/2413] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:51 2022] DELETE /identity/v3/projects/5ca0bb173adb4c04be1f0505b223cedc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:51.755883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:51.765891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74393800-74c8-4de9-b700-74cfb2da902f service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:51.819055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:51.819935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:51.820212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:51.820483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] PATH_INFO: `/v3/users/d8c01ac11d08478eb282931fd2062459` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:51.821692 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c01ac11d08478eb282931fd2062459)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:51.824213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:51.867429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c01ac11d08478eb282931fd2062459'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.868051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c01ac11d08478eb282931fd2062459'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.874739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c01ac11d08478eb282931fd2062459'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.882000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c01ac11d08478eb282931fd2062459'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.882685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c01ac11d08478eb282931fd2062459'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:51.899647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1213/2414] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:51 2022] DELETE /identity/v3/users/d8c01ac11d08478eb282931fd2062459 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:52.161408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:52.162849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:52.176023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4281b4f-8cbd-4239-bf28-7a23c289588a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:52.236960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:52.237831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:52.238116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:52.238395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] PATH_INFO: `/v3/projects/f382a0c12659435c894d537dbc9c60eb` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:52.239628 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f382a0c12659435c894d537dbc9c60eb)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:52.241690 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:52.286080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f382a0c12659435c894d537dbc9c60eb', 'request_id': 'req-00d7d601-06b2-40ed-af69-c88bc51d9e12'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.291184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f382a0c12659435c894d537dbc9c60eb', 'request_id': 'req-00d7d601-06b2-40ed-af69-c88bc51d9e12'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.299370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f382a0c12659435c894d537dbc9c60eb', 'request_id': 'req-00d7d601-06b2-40ed-af69-c88bc51d9e12'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.312054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] The token cache is being invalidate because project f382a0c12659435c894d537dbc9c60eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:52.312339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.313011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.314543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1202/2415] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:52 2022] DELETE /identity/v3/projects/f382a0c12659435c894d537dbc9c60eb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:52.318678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:52.320181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:52.333216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab896b-3f14-4f70-a5a1-da554536733b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:52.387069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:52.387922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:52.388198 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:52.388469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] PATH_INFO: `/v3/users/1ef1528105d14e0894a112fd91cc2772` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:52.389721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef1528105d14e0894a112fd91cc2772)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:52.391614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:52.434215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef1528105d14e0894a112fd91cc2772'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.434839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef1528105d14e0894a112fd91cc2772'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.441481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef1528105d14e0894a112fd91cc2772'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.448113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef1528105d14e0894a112fd91cc2772'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.448753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1e0feb60-5c1e-48dc-895e-a4f4e128c1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef1528105d14e0894a112fd91cc2772'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.465002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1214/2416] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:52 2022] DELETE /identity/v3/users/1ef1528105d14e0894a112fd91cc2772 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:52.720965 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:52.722503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:52.735532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7d601-06b2-40ed-af69-c88bc51d9e12 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:52.789081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:52.789935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:52.790228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:52.790479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] PATH_INFO: `/v3/projects/fd9e881bf76a4f4a881d924506c361f2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:52.792298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9e881bf76a4f4a881d924506c361f2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:52.794183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:52.839569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9e881bf76a4f4a881d924506c361f2', 'request_id': 'req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.845353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9e881bf76a4f4a881d924506c361f2', 'request_id': 'req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.851549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9e881bf76a4f4a881d924506c361f2', 'request_id': 'req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.864166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] The token cache is being invalidate because project fd9e881bf76a4f4a881d924506c361f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:52.864450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.865092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd32a2c2-5774-47c4-b1eb-e83c647f54ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:52.866589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1203/2417] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:52 2022] DELETE /identity/v3/projects/fd9e881bf76a4f4a881d924506c361f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:52.872594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:52.873127 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:52.873230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:52.891693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:52.892896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:52.950530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1215/2418] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:52.962428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:52.963154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:52.963154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:52.980984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:52.982153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:52.989790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1204/2419] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:52.993812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.004020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ec1ac-ca78-4a0f-bcdb-4fcab17d0c36 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.008520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.009196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.009476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.009747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.011330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:58:53.011647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.013310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.024448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1216/2420] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:58:53.028853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.037990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41060c-bc06-4335-8758-c64cb1dd9bb6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.042635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.043219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.043496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.043793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.044480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.046406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.071380 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1205/2421] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:53 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:53.076123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.085718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a36f50-682e-4ec4-b305-03983e26bbe6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.138996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.139648 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.139925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.140199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.140900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.142856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.163402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1217/2422] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:53 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:53.168471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.178954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22908901-f5f6-47fa-93c2-1dabb544fc74 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.184106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.184783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.185063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.185339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.186084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.187657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.197562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1206/2423] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:53.201590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.211082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d854-359f-4fb7-a5b2-f8edec6d8f38 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.215588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564721e-58bb-44c7-8179-ae1b5140a572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.216224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564721e-58bb-44c7-8179-ae1b5140a572 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.216499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564721e-58bb-44c7-8179-ae1b5140a572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.216803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564721e-58bb-44c7-8179-ae1b5140a572 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.217513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564721e-58bb-44c7-8179-ae1b5140a572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.219065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564721e-58bb-44c7-8179-ae1b5140a572 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.228402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1218/2424] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:53.233008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.242885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb756b-e19a-44a2-88df-1b6ffe5b36f8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.247539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.248404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.248712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.248987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] PATH_INFO: `/v3/projects/1c895757520146729a56b8098b59df25/users/a08c35bbf0b74d76ba6addd4c265be45/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.264016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c895757520146729a56b8098b59df25, user_id=a08c35bbf0b74d76ba6addd4c265be45, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.266709 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.279398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1207/2425] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:53 2022] PUT /identity/v3/projects/1c895757520146729a56b8098b59df25/users/a08c35bbf0b74d76ba6addd4c265be45/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 18:58:53.291750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.292039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.292310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.317355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] MFA Rules not processed for user `a08c35bbf0b74d76ba6addd4c265be45`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:53.318474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:53.380925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1219/2426] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:53.392564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:53.394255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:53.449811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37789ead-e61e-43eb-9a67-571ac98a4575 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:53.453391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcf089-1fd5-4389-a1fc-0ad3c81292a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:53.454021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcf089-1fd5-4389-a1fc-0ad3c81292a4 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:53.454356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcf089-1fd5-4389-a1fc-0ad3c81292a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:53.454584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcf089-1fd5-4389-a1fc-0ad3c81292a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:53.462499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcf089-1fd5-4389-a1fc-0ad3c81292a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:53.464508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcf089-1fd5-4389-a1fc-0ad3c81292a4 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:53.472973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1208/2427] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:56.404697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:56.463502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a97be8-ae39-4276-97fa-2e1c74442e8a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:56.467934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:56.468809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:56.469090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:56.469363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] PATH_INFO: `/v3/users/a08c35bbf0b74d76ba6addd4c265be45` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:56.470633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08c35bbf0b74d76ba6addd4c265be45)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:56.472538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:56.516248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08c35bbf0b74d76ba6addd4c265be45'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:56.516913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08c35bbf0b74d76ba6addd4c265be45'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:56.523518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08c35bbf0b74d76ba6addd4c265be45'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:56.530374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08c35bbf0b74d76ba6addd4c265be45'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:56.531324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08c35bbf0b74d76ba6addd4c265be45'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:56.547591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1220/2428] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:58:56 2022] DELETE /identity/v3/users/a08c35bbf0b74d76ba6addd4c265be45 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:56.552151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:56.552609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:56.552819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:56.569972 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:56.571158 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:56.627512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1209/2429] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:56.637853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:56.639164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:56.693003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939fb35-6760-4576-823b-6940701fd28d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:56.696538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c155e-3340-4811-9b7f-79362d7e262a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:56.697188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c155e-3340-4811-9b7f-79362d7e262a service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:56.697510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c155e-3340-4811-9b7f-79362d7e262a service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:56.697777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c155e-3340-4811-9b7f-79362d7e262a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:56.706855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c155e-3340-4811-9b7f-79362d7e262a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:56.708247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c155e-3340-4811-9b7f-79362d7e262a service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:56.717203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1221/2430] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:58:56.970953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:56.972426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:56.984432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d7642-b969-4f96-b8a0-270c175eb80d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:56.989012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:56.989858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:56.990134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:56.990402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] PATH_INFO: `/v3/projects/1c895757520146729a56b8098b59df25` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:56.991564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c895757520146729a56b8098b59df25)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:56.993485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.038595 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c895757520146729a56b8098b59df25', 'request_id': 'req-66439cde-7f9d-4e22-8110-cd080a8edfdf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:57.044270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c895757520146729a56b8098b59df25', 'request_id': 'req-66439cde-7f9d-4e22-8110-cd080a8edfdf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:57.050216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c895757520146729a56b8098b59df25', 'request_id': 'req-66439cde-7f9d-4e22-8110-cd080a8edfdf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:57.062848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] The token cache is being invalidate because project 1c895757520146729a56b8098b59df25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:58:57.063130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:57.063744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-66439cde-7f9d-4e22-8110-cd080a8edfdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:58:57.065291 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1210/2431] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:58:56 2022] DELETE /identity/v3/projects/1c895757520146729a56b8098b59df25 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:57.070288 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.070580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.070885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.087972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:57.089194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:57.146030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1222/2432] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:57.153731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.154024 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.154305 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.173643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:57.174822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:57.183160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1211/2433] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:57.187269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.196709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9125a30-1998-40ba-8da3-a83305a8b981 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.201149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.201816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.202092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.202362 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.203068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:58:57.203369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.204970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.216371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1223/2434] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 18:58:57.220361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.229816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71390f34-c2d8-4dc5-a4c9-56fe1882cc2e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.234376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.235005 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.235285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.235556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.236225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.238143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.262980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1212/2435] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:57.267092 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.277360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511d1f0-3370-407d-b659-69ec222430f2 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.331443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.332090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.332370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.332667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.333351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.335242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.356006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1224/2436] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:57.360016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.370012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75850243-f476-40c8-b39a-4b069b87424c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.374567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.375274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.375557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.375831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.376541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.378151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.388304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1213/2437] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:57.391743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.401740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab290eaf-ab62-48d4-81d5-1bd649384793 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.406492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61559-135a-49be-860a-40dbbfda1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.407121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61559-135a-49be-860a-40dbbfda1f27 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.407395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61559-135a-49be-860a-40dbbfda1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.407663 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61559-135a-49be-860a-40dbbfda1f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.408365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61559-135a-49be-860a-40dbbfda1f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.409955 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61559-135a-49be-860a-40dbbfda1f27 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.419821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1225/2438] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:57.423611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.433084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd127426-2397-4d54-a757-40daf1ba6526 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.437590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.438442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.438717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.438987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] PATH_INFO: `/v3/projects/7e551ea63b3e4042a0f300f6572aaf15/users/b4b59082bb7843389b03b1970df6b861/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.452773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e551ea63b3e4042a0f300f6572aaf15, user_id=b4b59082bb7843389b03b1970df6b861, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.455385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.468405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1214/2439] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:57 2022] PUT /identity/v3/projects/7e551ea63b3e4042a0f300f6572aaf15/users/b4b59082bb7843389b03b1970df6b861/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:57.479564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.479849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.480122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.505049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] MFA Rules not processed for user `b4b59082bb7843389b03b1970df6b861`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:57.506237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:57.566143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1226/2440] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:57.570532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.628390 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa76151-0095-44e9-af49-1abbda548d38 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.632995 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.633637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.633918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.634191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.635371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.637301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.661668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1215/2441] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:57.665668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.676196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99425d0-eef3-4d04-9ac0-37aae91d3ee5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.730403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.731054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.731330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.731595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.732276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.734193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.753227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1227/2442] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:57.757267 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.767662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849eb3af-acfd-4e68-bb2d-70ddff96d92d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.772199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.772885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.773165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.773440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.774150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.776334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.786738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1216/2443] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:57.790414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.799939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798bd4f-c90f-4843-99cb-7fcbb13506fb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.804445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef77cf-f2db-4360-8241-326f20ce38d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.805141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef77cf-f2db-4360-8241-326f20ce38d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.805418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef77cf-f2db-4360-8241-326f20ce38d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.805687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef77cf-f2db-4360-8241-326f20ce38d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.806421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef77cf-f2db-4360-8241-326f20ce38d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.807980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef77cf-f2db-4360-8241-326f20ce38d5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.817335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1228/2444] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:57.821282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:57.831311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90cea8-4a18-4203-b7fd-44f76bc886c3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:57.836018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:57.836903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.837182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.837453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] PATH_INFO: `/v3/projects/d75a878f0c1c4d00979344fe411d31df/users/da083ba6875144e2bf0b1b91fd9906a4/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.852827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75a878f0c1c4d00979344fe411d31df, user_id=da083ba6875144e2bf0b1b91fd9906a4, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:57.855474 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:57.868041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1217/2445] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:57 2022] PUT /identity/v3/projects/d75a878f0c1c4d00979344fe411d31df/users/da083ba6875144e2bf0b1b91fd9906a4/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:57.878084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:57.878399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:57.878676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:57.904601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] MFA Rules not processed for user `da083ba6875144e2bf0b1b91fd9906a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:57.906080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:57.968594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1229/2446] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:58:57.973254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.032488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab2059-4cb2-4a7c-8bc3-3d643116e067 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.037159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.037808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.038085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.038356 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.039045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.040976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.064684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1218/2447] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:58:57 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:58:58.069006 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.079764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d83280-a69d-41ae-9637-00ac500cad3e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.132624 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.133411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.133686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.133954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.134630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.137115 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.156412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1230/2448] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:58:58 2022] POST /identity/v3/users => generated 299 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 18:58:58.160459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.172013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca69c5b-adc2-42f3-857b-6217899e8be1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.176835 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.177492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.177773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.178048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.178769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.180350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.200533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1219/2449] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:58.204652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.214377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee16783-aadb-44b6-bf66-198668be5ea0 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.218804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.219652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.219929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.220206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] PATH_INFO: `/v3/projects/5b3f07cfe00a4a09b3df5108cf7fd617/users/2963396e0dd341c384aa9687de3b4689/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.234021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3f07cfe00a4a09b3df5108cf7fd617, user_id=2963396e0dd341c384aa9687de3b4689, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.236612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.249595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1231/2450] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:58 2022] PUT /identity/v3/projects/5b3f07cfe00a4a09b3df5108cf7fd617/users/2963396e0dd341c384aa9687de3b4689/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:58.253310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.263952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07a40-f0f1-4485-8367-904d04d5bf29 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.316530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25e429-a96e-4cc4-9dcd-e70a7bb1cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.317202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e429-a96e-4cc4-9dcd-e70a7bb1cac5 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.317479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e429-a96e-4cc4-9dcd-e70a7bb1cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.317750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25e429-a96e-4cc4-9dcd-e70a7bb1cac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.318450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25e429-a96e-4cc4-9dcd-e70a7bb1cac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.320017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25e429-a96e-4cc4-9dcd-e70a7bb1cac5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.330014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1220/2451] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:58:58.334022 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.344314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54aca2-bdb5-4e22-9dab-54cfc765e706 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.349134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.350075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.350346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.350614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] PATH_INFO: `/v3/projects/5b3f07cfe00a4a09b3df5108cf7fd617/users/2963396e0dd341c384aa9687de3b4689/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.352565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3f07cfe00a4a09b3df5108cf7fd617, user_id=2963396e0dd341c384aa9687de3b4689, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.355816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.366608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1232/2452] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:58:58 2022] PUT /identity/v3/projects/5b3f07cfe00a4a09b3df5108cf7fd617/users/2963396e0dd341c384aa9687de3b4689/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:58:58.378171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.378582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.378865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.405542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] MFA Rules not processed for user `2963396e0dd341c384aa9687de3b4689`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:58:58.407136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:58.477794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1221/2453] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:58:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 18:58:58.543295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:58:58.545035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:58:58.602402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b5d6c-b2aa-4374-96b9-b30890b44472 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:58:58.606480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:58:58.607111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:58:58.607386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:58:58.607657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:58:58.660574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:58:58.662031 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:58:58.670982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1233/2454] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:58:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:59:00.128946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:00.130317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:00.141926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449174b6-4511-4cf3-a5d9-de3fe7f35ca1 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:00.145406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:00.146033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:00.146311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:00.146580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:00.200037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:00.201421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:00.210601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1222/2455] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:59:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:59:00.697786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:00.706004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b591dc4-2f5b-4fb5-8083-7f159c7c89b6 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:00.709588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cf4d-cff6-48cd-8894-b1aa1b196e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:00.710207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cf4d-cff6-48cd-8894-b1aa1b196e7e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:00.710480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cf4d-cff6-48cd-8894-b1aa1b196e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:00.710747 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cf4d-cff6-48cd-8894-b1aa1b196e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:00.720032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07cf4d-cff6-48cd-8894-b1aa1b196e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:00.721386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07cf4d-cff6-48cd-8894-b1aa1b196e7e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:00.730323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1234/2456] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:59:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:59:04.523308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:04.582750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adbbca-f5f9-42a3-8f68-d453ee8b9e87 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:04.587242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:04.588100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:04.588381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:04.588680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] PATH_INFO: `/v3/users/b4b59082bb7843389b03b1970df6b861` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:04.589849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b59082bb7843389b03b1970df6b861)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:04.591764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:04.634786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b59082bb7843389b03b1970df6b861'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:04.641235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b59082bb7843389b03b1970df6b861'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:04.641871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b59082bb7843389b03b1970df6b861'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:04.648412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b59082bb7843389b03b1970df6b861'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:04.649034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b59082bb7843389b03b1970df6b861'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:04.666587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1223/2457] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:59:04 2022] DELETE /identity/v3/users/b4b59082bb7843389b03b1970df6b861 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:04.671314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:04.671785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:04.671941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:04.689350 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:04.690520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:04.747965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1235/2458] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:59:04.757366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:04.758739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:04.813280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5a3e6-0298-4028-9e7f-db3e2e7475ec admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:04.816899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:04.817539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:04.817815 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:04.818081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:04.827270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:04.828711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:04.837774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1224/2459] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:59:05.099757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:05.101302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:05.113478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f61b07-6350-4f32-8f0e-025b654b5fc1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:05.118073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:05.119497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:05.119779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:05.120051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] PATH_INFO: `/v3/projects/7e551ea63b3e4042a0f300f6572aaf15` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:05.121257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e551ea63b3e4042a0f300f6572aaf15)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:05.123162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:05.170238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e551ea63b3e4042a0f300f6572aaf15', 'request_id': 'req-d2e3df23-83d9-483c-99d1-284abd0e3aab'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.175502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e551ea63b3e4042a0f300f6572aaf15', 'request_id': 'req-d2e3df23-83d9-483c-99d1-284abd0e3aab'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.182232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e551ea63b3e4042a0f300f6572aaf15', 'request_id': 'req-d2e3df23-83d9-483c-99d1-284abd0e3aab'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.192840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] The token cache is being invalidate because project 7e551ea63b3e4042a0f300f6572aaf15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:59:05.193121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.193743 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.195268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1236/2460] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:59:05 2022] DELETE /identity/v3/projects/7e551ea63b3e4042a0f300f6572aaf15 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:05.200743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:05.212826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80943ab-52a3-45b2-8e74-8f4d115f633d service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:05.270969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:05.271829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:05.272110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:05.272383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] PATH_INFO: `/v3/users/da083ba6875144e2bf0b1b91fd9906a4` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:05.273576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da083ba6875144e2bf0b1b91fd9906a4)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:05.276173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:05.318465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da083ba6875144e2bf0b1b91fd9906a4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.325410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da083ba6875144e2bf0b1b91fd9906a4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.326051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da083ba6875144e2bf0b1b91fd9906a4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.332999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da083ba6875144e2bf0b1b91fd9906a4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.333650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da083ba6875144e2bf0b1b91fd9906a4'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.350752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1225/2461] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:59:05 2022] DELETE /identity/v3/users/da083ba6875144e2bf0b1b91fd9906a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:05.605807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:05.607295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:05.620132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3df23-83d9-483c-99d1-284abd0e3aab admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:05.674014 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:05.674889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:05.675164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:05.675468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] PATH_INFO: `/v3/projects/d75a878f0c1c4d00979344fe411d31df` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:05.676652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75a878f0c1c4d00979344fe411d31df)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:05.678536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:05.721770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75a878f0c1c4d00979344fe411d31df', 'request_id': 'req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.726832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75a878f0c1c4d00979344fe411d31df', 'request_id': 'req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.733730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75a878f0c1c4d00979344fe411d31df', 'request_id': 'req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.744468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] The token cache is being invalidate because project d75a878f0c1c4d00979344fe411d31df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:59:05.744720 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.745332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.746812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1237/2462] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:59:05 2022] DELETE /identity/v3/projects/d75a878f0c1c4d00979344fe411d31df => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:05.750867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:05.752266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:05.765313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992e19c-e21b-4c3e-80a6-503c8ecfac84 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:05.818668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:05.819457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:05.819733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:05.820004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] PATH_INFO: `/v3/users/2963396e0dd341c384aa9687de3b4689` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:05.821180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2963396e0dd341c384aa9687de3b4689)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:05.823615 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:05.866684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2963396e0dd341c384aa9687de3b4689'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.874430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2963396e0dd341c384aa9687de3b4689'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.875094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2963396e0dd341c384aa9687de3b4689'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.882302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2963396e0dd341c384aa9687de3b4689'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.883111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7014b2cc-f1ad-401e-9ee9-c5ebdc984ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2963396e0dd341c384aa9687de3b4689'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:05.899325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1226/2463] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 18:59:05 2022] DELETE /identity/v3/users/2963396e0dd341c384aa9687de3b4689 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:06.157563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.158992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:06.174826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4be1b-5d2f-40ac-a05d-e90fb2dd4e71 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.229894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.230744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.231027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.231305 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] PATH_INFO: `/v3/projects/5b3f07cfe00a4a09b3df5108cf7fd617` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.232467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b3f07cfe00a4a09b3df5108cf7fd617)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.234401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.289099 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b3f07cfe00a4a09b3df5108cf7fd617', 'request_id': 'req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:06.294322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b3f07cfe00a4a09b3df5108cf7fd617', 'request_id': 'req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:06.301434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b3f07cfe00a4a09b3df5108cf7fd617', 'request_id': 'req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:06.312838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] The token cache is being invalidate because project 5b3f07cfe00a4a09b3df5108cf7fd617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 18:59:06.313121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:06.313759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fbe87085-427b-4578-ba2d-8e0a6e4a17f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 18:59:06.315280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1238/2464] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 18:59:06 2022] DELETE /identity/v3/projects/5b3f07cfe00a4a09b3df5108cf7fd617 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:06.320263 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.320552 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.320876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.339176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:06.340307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:06.396387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1227/2465] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:59:06.403929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.404286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.404649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.422707 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:06.423880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:06.432173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1239/2466] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:59:06.436038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.445301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df815a2-491c-4e71-aa7e-fef99bb069b9 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.450387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.451026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.451306 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.451585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.452301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 18:59:06.452606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.454212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.465157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1228/2467] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 18:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 18:59:06.469197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.477997 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65510511-eaaf-4034-ac65-0ba822c4707a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.482579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.483211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.483490 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.483761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.484448 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.486366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.512343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1240/2468] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 18:59:06 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:59:06.516678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.526438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ffac-63ea-4783-84f9-c451792c303a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.582556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.583245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.583524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.583798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.584475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.586414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.607094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1229/2469] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 18:59:06 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 18:59:06.611280 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.622269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c02179-9bba-4977-b824-2bd35930257d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.627145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.627859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.628137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.628137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.628137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.628137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.641693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1241/2470] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:59:06.646748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.657429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea63c7-4d78-425b-b987-757fd3f35391 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.662189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86b961-fa41-4559-bae3-718414784a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.662886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86b961-fa41-4559-bae3-718414784a89 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.663170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86b961-fa41-4559-bae3-718414784a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.663465 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86b961-fa41-4559-bae3-718414784a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.664218 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86b961-fa41-4559-bae3-718414784a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.666399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86b961-fa41-4559-bae3-718414784a89 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.676293 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1230/2471] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 18:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 18:59:06.680404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.691104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507d300-5279-4665-aad3-8618790cce42 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.696062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.697104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.697391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.697649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] PATH_INFO: `/v3/projects/9be561020bdd441096d407f87ac63011/users/9a7f9e2344fa4cfab4e6836d634865ad/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.711399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be561020bdd441096d407f87ac63011, user_id=9a7f9e2344fa4cfab4e6836d634865ad, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.714045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.727134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1242/2472] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 18:59:06 2022] PUT /identity/v3/projects/9be561020bdd441096d407f87ac63011/users/9a7f9e2344fa4cfab4e6836d634865ad/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 18:59:06.732696 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b0d7-1b62-4d37-a812-cca026580e5f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.732991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b0d7-1b62-4d37-a812-cca026580e5f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.733310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b0d7-1b62-4d37-a812-cca026580e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.751324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-dc77b0d7-1b62-4d37-a812-cca026580e5f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:06.752507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-dc77b0d7-1b62-4d37-a812-cca026580e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:06.808205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1231/2473] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:59:06.817716 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:06.819377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:06.872523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28776408-1112-450b-8f4d-7e09a5037e05 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:06.876012 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:06.876656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:06.876936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:06.877210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:06.885796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:06.887183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:06.896454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1243/2474] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:59:07.471576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ba137-aa59-4932-afee-8d257c2e8f55 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:07.471883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ba137-aa59-4932-afee-8d257c2e8f55 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:07.472156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ba137-aa59-4932-afee-8d257c2e8f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:07.497094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0f8ba137-aa59-4932-afee-8d257c2e8f55 None None] MFA Rules not processed for user `9a7f9e2344fa4cfab4e6836d634865ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:07.498275 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0f8ba137-aa59-4932-afee-8d257c2e8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:07.562852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1232/2475] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:59:07.574120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 18:59:07.575486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:07.629988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545b779-c115-4e49-ad8e-a84f40ae3cfd service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 18:59:07.633569 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6ec44-7df9-48cc-b072-e94e8b6f0651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 18:59:07.635483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6ec44-7df9-48cc-b072-e94e8b6f0651 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:07.635777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6ec44-7df9-48cc-b072-e94e8b6f0651 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:07.636052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6ec44-7df9-48cc-b072-e94e8b6f0651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:07.644395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6ec44-7df9-48cc-b072-e94e8b6f0651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 18:59:07.645855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6ec44-7df9-48cc-b072-e94e8b6f0651 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 18:59:07.655038 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1244/2476] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 18:59:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 18:59:08.984280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:08.984827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:08.984925 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:09.001693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] MFA Rules not processed for user `9a7f9e2344fa4cfab4e6836d634865ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:09.002849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:09.009990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1233/2477] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 18:59:36.634717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec754-d70b-4776-baa5-1b9cdd652dc7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 18:59:36.635236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec754-d70b-4776-baa5-1b9cdd652dc7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 18:59:36.635333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec754-d70b-4776-baa5-1b9cdd652dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 18:59:36.652186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-20fec754-d70b-4776-baa5-1b9cdd652dc7 None None] MFA Rules not processed for user `8fec0b9b99934e90b5c98eb43ca4a448`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 18:59:36.653393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-20fec754-d70b-4776-baa5-1b9cdd652dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 18:59:36.706208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1245/2478] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:00:06.071783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:00:06.073352 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:00:06.140387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa15b-d830-4a85-9219-5d54a601fe71 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:00:06.145381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:00:06.146018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:00:06.146296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:00:06.146567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:00:06.149041 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:00:06.160440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:00:06.162058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:00:06.171757 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1234/2479] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:00:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:00:21.204044 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60984821-a0f3-4ed6-8c6f-0390a5ce0476 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:00:21.204550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60984821-a0f3-4ed6-8c6f-0390a5ce0476 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:00:21.204686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60984821-a0f3-4ed6-8c6f-0390a5ce0476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:00:21.222811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-60984821-a0f3-4ed6-8c6f-0390a5ce0476 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:00:21.223989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-60984821-a0f3-4ed6-8c6f-0390a5ce0476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:00:21.277363 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1246/2480] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 19:00:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:00:21.322977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:00:21.324395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:00:21.389990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48009f-618e-4a8f-9a69-f29296a9397c service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:00:21.394946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e753a-38b7-46df-b770-71771997e9b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:00:21.395589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e753a-38b7-46df-b770-71771997e9b7 service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:00:21.395871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e753a-38b7-46df-b770-71771997e9b7 service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:00:21.396177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e753a-38b7-46df-b770-71771997e9b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:00:21.404105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e753a-38b7-46df-b770-71771997e9b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:00:21.405989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e753a-38b7-46df-b770-71771997e9b7 service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:00:21.413791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1235/2481] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:00:32.788548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:00:32.789112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:00:32.789230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:00:32.806307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] MFA Rules not processed for user `9a7f9e2344fa4cfab4e6836d634865ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:00:32.807413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:00:32.814112 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1247/2482] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:00:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:00:41.748396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:00:41.748885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:00:41.748997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:00:41.768668 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:00:41.769964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:00:41.778146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1236/2483] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 19:00:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:00:41.793100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:00:41.805936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5baf69-dd7b-49c8-9779-ead82dc6b4e4 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:00:41.811076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:00:41.811733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:00:41.812039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:00:41.812319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:00:41.822187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:00:41.823812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:00:41.833379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1248/2484] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:00:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:01:01.842273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:01:01.843724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:01:01.855060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be4acc-4f0d-4644-bf7a-9e7d9f0b639d None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:01:01.858544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:01:01.859174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:01:01.859452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:01:01.859723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:01:01.861439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:01:01.930777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:01:01.932202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca219c-ff05-4c15-9389-21f9c8636e14 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:01:01.943220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1237/2485] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:01:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:01:06.011759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:01:06.021087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9056594-1d94-44fb-89f9-99b7df175ba3 service swift] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:01:06.024582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:01:06.025238 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:01:06.025549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:01:06.025828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:01:06.027607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:01:06.042711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:01:06.044170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:01:06.056283 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1249/2486] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:01:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:01:13.870273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63db89-c323-450d-902f-976913bcc85a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:01:13.870770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63db89-c323-450d-902f-976913bcc85a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:01:13.870859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63db89-c323-450d-902f-976913bcc85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:01:13.888917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8c63db89-c323-450d-902f-976913bcc85a None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:01:13.890120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8c63db89-c323-450d-902f-976913bcc85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:01:13.897419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1238/2487] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 16 19:01:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:01:13.912440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:01:13.926670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73e054-a7b8-49da-af88-707b709374cf service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:01:13.931278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:01:13.931918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:01:13.932205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:01:13.932623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:01:13.942219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:01:13.944512 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:01:13.953451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1250/2488] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:02:08.889139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63db89-c323-450d-902f-976913bcc85a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:02:08.890542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8c63db89-c323-450d-902f-976913bcc85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:08.901720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63db89-c323-450d-902f-976913bcc85a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:02:08.905304 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:02:08.905939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:02:08.906220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:02:08.906494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:02:08.908482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:08.985364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:02:08.986795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:02:09.005060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1239/2489] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:02:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:02:10.614206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:02:10.623356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44080ed-ab5c-409d-a3d6-9c24690f2129 service swift] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:02:10.627153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:02:10.627763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:02:10.628044 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:02:10.628320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:02:10.630587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:10.708110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:02:10.709497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:02:10.727637 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1251/2490] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:02:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:02:22.205154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:02:22.214282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6aa0b-c131-44ef-a9b2-aa44a44ffdec service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:02:22.217804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:02:22.218438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:02:22.218717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:02:22.218989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:02:22.220766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:22.230166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:02:22.231573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:02:22.239334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1240/2491] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:02:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:02:26.157442 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:02:26.171575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40530da-921c-4a44-9a67-caddf70fad00 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:02:26.175951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:02:26.178117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:02:26.179519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:02:26.180033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:02:26.183508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:26.261001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:02:26.262471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:02:26.322105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1252/2492] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:02:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:02:59.140955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:02:59.142343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:59.157738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b85bb-c990-4a04-b331-f4d34f4b14e2 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:02:59.163881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:02:59.164505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:02:59.164824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:02:59.165098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:02:59.166771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:02:59.221345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:02:59.222842 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:02:59.231983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1241/2493] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:02:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:04:26.409848 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:04:26.446004 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c24dc-2009-45b8-9216-4540ac99ca54 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:04:26.449581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:04:26.450204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:04:26.450484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:04:26.450768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:04:26.470989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:04:26.472397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:04:26.480906 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1253/2494] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:04:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:06:01.009075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:06:01.036365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e07c-b50d-43b8-8bda-91191c60b1a1 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:06:01.040735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:06:01.041366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:06:01.041665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:06:01.041923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:06:01.065586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:06:01.067036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:06:01.079388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1242/2495] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:06:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:06:06.804866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:06:06.815137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb079fc1-9630-4725-8eaa-6225756814bc service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:06:06.819677 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:06:06.820249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:06:06.820527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:06:06.820828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:06:06.830683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:06:06.832082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:06:06.852454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1254/2496] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:06:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:06:07.580813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:06:07.582179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:06:07.626775 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d49ef-2451-42a6-9b73-6a7f236cbdde service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:06:07.631840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:06:07.632472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:06:07.632783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:06:07.633061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:06:07.641283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:06:07.642854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:06:07.652372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1243/2497] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:06:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:07:59.265729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:07:59.267093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:07:59.292417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085e1ea-0e64-4ff8-b339-911c7d9f3b69 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:07:59.297778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:07:59.298395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:07:59.298673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:07:59.298945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:07:59.306827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:07:59.308377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:07:59.333914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1255/2498] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:07:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:08:17.252991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:08:17.262748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec280b9-0a31-4464-8458-1f3a49432658 service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:08:17.266280 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:08:17.266904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:08:17.267182 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:08:17.267455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:08:17.274437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:08:17.275851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:08:17.283538 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1244/2499] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:08:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:08:27.067761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:08:27.068834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:08:27.076836 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0ff69-9ee3-4092-99c9-3a6531802848 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:08:27.078227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2545f-1c08-41fc-997e-cea5610b8017 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:08:27.080279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:08:27.080931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:08:27.081866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:08:27.082139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:08:27.083022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:08:27.083720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:08:27.084071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:08:27.084443 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:08:27.090801 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:08:27.092230 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:08:27.093069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:08:27.094581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:08:27.101257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1256/2500] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:08:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:08:27.103821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1245/2501] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:08:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:08:27.143166 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:08:27.152186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a53a56-c884-4bfc-a5b8-f73c786163a3 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:08:27.155846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:08:27.156581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:08:27.156915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:08:27.157190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:08:27.169444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:08:27.171084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:08:27.180863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1257/2502] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:08:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:08:29.079740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:08:29.095172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69a418-fef2-4fb4-97d5-2db3963e0c54 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:08:29.102220 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:08:29.103567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:08:29.103923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:08:29.104192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:08:29.106507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:08:29.119577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:08:29.121135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:08:29.132741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1246/2503] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:08:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:09:27.233713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:09:27.246389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8c6da-bebf-4e4c-9d35-a6dd16d7b066 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:09:27.251623 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:09:27.252243 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:09:27.252519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:09:27.252822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:09:27.254638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:09:27.329549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:09:27.331116 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:09:27.348650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1258/2504] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:09:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:11:10.505332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:11:10.569170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234165c-7ce0-453e-b57e-7ff5ab713ced service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:11:10.574614 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:11:10.575250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:11:10.575533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:11:10.575810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:11:10.577725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:11:10.589262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:11:10.590833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:11:10.600399 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1247/2505] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:11:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:12:27.476484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:12:27.476484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:12:27.514164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:12:27.515728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:12:27.543080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e421-ecf0-4d87-b429-2def4edcd9d9 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:12:27.546760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:12:27.547390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:12:27.547665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:12:27.547961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:12:27.549839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:12:27.569860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3464db3-51c4-471e-bc33-841cde0bb010 service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:12:27.573457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:12:27.574119 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:12:27.574391 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:12:27.574981 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:12:27.576492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:12:27.639559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:12:27.641599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:12:27.653536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1259/2506] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:12:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:12:27.668464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:12:27.669929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:12:27.682106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1248/2507] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:12:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:12:59.732977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:12:59.741406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a73ba-facf-44a8-beee-8f4629cfa328 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:12:59.745029 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:12:59.745651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:12:59.745928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:12:59.746200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:12:59.748016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:12:59.814979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:12:59.816391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:12:59.825499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1260/2508] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:12:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:13:23.238766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:13:23.240347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:13:23.307880 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7758159-0fd2-4f66-b2a2-5dcb79134b05 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:13:23.311495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:13:23.312134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:13:23.312423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:13:23.312729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:13:23.314582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:13:23.323655 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:13:23.325128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:13:23.332882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1249/2509] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:13:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:13:26.395918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:13:26.404847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd053c-efbe-4a37-b6a4-7bc27c7073ce service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:13:26.408369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:13:26.409036 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:13:26.409315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:13:26.409589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:13:26.411400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:13:26.488074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:13:26.489451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:13:26.506767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1261/2510] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:13:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:13:27.257584 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:13:27.259212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:13:27.290502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfca05-b0ba-4c6b-b89e-0e95affb86a0 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:13:27.298430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:13:27.299067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:13:27.299346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:13:27.299620 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:13:27.301609 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:13:27.377432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:13:27.379039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:13:27.396930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1250/2511] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:13:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:13:28.322222 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:13:28.330475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dd00c-cd34-446c-95b6-2ff06f6d3ec6 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:13:28.334050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:13:28.334675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:13:28.334954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:13:28.335226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:13:28.337161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:13:28.413239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:13:28.414651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:13:28.433232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1262/2512] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:13:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:14:26.131857 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:14:26.141488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78034018-d9fd-42dc-87f4-d968336d9129 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:14:26.144979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:14:26.145605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:14:26.145883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:14:26.146152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:14:26.154671 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:14:26.156062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:14:26.164791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1251/2513] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:16:14.545653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:16:14.548844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:16:14.637211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a007e96-a512-4d2c-ae47-aeb8f735c26b service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:16:14.642119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:16:14.642748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:16:14.643025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:16:14.643296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:16:14.652189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:16:14.653788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:16:14.663225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1263/2514] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:17:58.588311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:17:58.616177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94d5ef-c513-4e36-b516-36d59b44f537 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:17:58.619951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:17:58.620583 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:17:58.620905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:17:58.621180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:17:58.629419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:17:58.630872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:17:58.639662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1252/2515] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:17:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:26.115742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:26.124947 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef532-b4c9-4ba0-bf21-5bb9c4ade620 service placement] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:26.128466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:26.129131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:26.129416 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:26.129690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:26.138500 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:26.139905 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:26.148599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1264/2516] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:26.243989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:26.261977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dbfa2-75b4-4ba8-b77f-d422fcdfe82c service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:26.265787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:26.266418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:26.266719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:26.266989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:26.274413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:26.275860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:26.283660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1253/2517] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:26.899207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:26.900765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:26.943674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb27f-7aa4-4745-a93b-2d2cee4918da service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:26.949075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:26.949697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:26.950020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:26.950244 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:26.957844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:26.959963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:26.969709 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1265/2518] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:28.220217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:28.229859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec83c47-8736-412d-a624-55fec43fd453 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:28.233427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:28.234057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:28.234336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:28.234610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:28.243381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:28.244821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:28.253673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1254/2519] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:28.390941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:28.392467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:28.404259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211e469-255e-43fc-bb39-be9a3b09f4dd service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:28.407762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:28.408379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:28.408682 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:28.408956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:28.410651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:28.414048 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:28.423732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e126ef-2043-42b8-80e2-15f23e4489a9 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:28.427205 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:28.427828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:28.428128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:28.428401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:28.439140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:28.440557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:28.451846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1255/2520] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:18:28.493089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:28.494438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:28.513314 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1266/2521] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:29.661074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:29.671514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8f5b-017b-4130-8103-9d4897ec4ae2 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:29.675808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:29.677247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:29.677599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:29.677938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:29.679831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:29.693001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:29.694624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:29.705115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1256/2522] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:30.905139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:30.926551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb374bb8-b76f-400e-b6ce-cb59a89ceaa2 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:30.936883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:30.938403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:30.939030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:30.939559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:30.960841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:30.970354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:30.987232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1267/2523] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:18:31.920258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:31.925331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:31.977954 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e713f-3501-4cff-97d0-8a834014504e service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:31.993686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:31.994905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:31.995430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:31.995712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] PATH_INFO: `/v3/users/867f26290c254777b65cb739263d708a` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:31.996991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=867f26290c254777b65cb739263d708a)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:31.999272 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:32.051730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867f26290c254777b65cb739263d708a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.066463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867f26290c254777b65cb739263d708a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.067455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867f26290c254777b65cb739263d708a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.080859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867f26290c254777b65cb739263d708a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.081591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-daa5b780-2b93-40b1-915e-83bc051d2725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867f26290c254777b65cb739263d708a'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.104673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1257/2524] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:31 2022] DELETE /identity/v3/users/867f26290c254777b65cb739263d708a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:32.303686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:32.316155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70421d-6a58-44b1-ad25-d6551d12470b service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:32.395257 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:32.396165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.396443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.396745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] PATH_INFO: `/v3/projects/0d6d170d3db6427f867c5c7caaf295c2` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.397935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6d170d3db6427f867c5c7caaf295c2)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:32.400845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:32.468761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6d170d3db6427f867c5c7caaf295c2', 'request_id': 'req-e71da2b2-8832-478d-8316-53537d752ca7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.476052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6d170d3db6427f867c5c7caaf295c2', 'request_id': 'req-e71da2b2-8832-478d-8316-53537d752ca7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.485534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6d170d3db6427f867c5c7caaf295c2', 'request_id': 'req-e71da2b2-8832-478d-8316-53537d752ca7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.501502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] The token cache is being invalidate because project 0d6d170d3db6427f867c5c7caaf295c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:32.501792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.502444 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e71da2b2-8832-478d-8316-53537d752ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:32.504378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1268/2525] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:32 2022] DELETE /identity/v3/projects/0d6d170d3db6427f867c5c7caaf295c2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:18:32.510432 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.510745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.511023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.571847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:32.573265 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:32.647098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1258/2526] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:18:32.655868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.656237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.656577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.685549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:32.686771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:32.695386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1269/2527] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:32.699366 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:32.710823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddd1e4-b349-401c-8363-e73eefca7a1f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:32.716876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:32.717743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.718302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.718728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.720088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:18:32.720567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:32.722370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:32.735962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1259/2528] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:18:32.741187 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:32.750665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7635e07-74cd-4520-a22b-42edf863632a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:32.756525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:32.757555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.758061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.760809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.761629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:32.763778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:32.829678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1270/2529] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:18:32 2022] POST /identity/v3/projects => generated 367 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:32.834160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:32.844910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b43bc7-b966-418c-a988-4e31a57318f1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:32.921770 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:32.922528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.923035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.923316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.924015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:32.925971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:32.947557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1260/2530] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:18:32 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:32.951734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:32.966266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d02b7-2a5f-428f-b4e5-90f1df9224e8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:32.971571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:32.972227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:32.972503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:32.972818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:32.973534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:32.976236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:32.988608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1271/2531] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:32.993616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.006081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1facaf7-5f56-4448-9329-306814f51d76 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.015474 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5fb3d-02b2-4fdd-9ec9-224946deec4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.016242 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5fb3d-02b2-4fdd-9ec9-224946deec4f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.019862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5fb3d-02b2-4fdd-9ec9-224946deec4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.020415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5fb3d-02b2-4fdd-9ec9-224946deec4f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.022144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5fb3d-02b2-4fdd-9ec9-224946deec4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.024085 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5fb3d-02b2-4fdd-9ec9-224946deec4f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.035751 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1261/2532] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:33.039666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.051922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ed348-ca60-4bec-b67a-848440f100d4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.057741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.058700 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.059052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.059418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] PATH_INFO: `/v3/projects/6d2ee49f8ed843afbdb544ecb7b8fc7e/users/828ecca046c0412aacf39d4d94b900ad/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.077002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2ee49f8ed843afbdb544ecb7b8fc7e, user_id=828ecca046c0412aacf39d4d94b900ad, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.079547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.097681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1272/2533] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:33 2022] PUT /identity/v3/projects/6d2ee49f8ed843afbdb544ecb7b8fc7e/users/828ecca046c0412aacf39d4d94b900ad/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:33.108652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.109311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.109807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.163311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] MFA Rules not processed for user `828ecca046c0412aacf39d4d94b900ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:33.164545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:33.238631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1262/2534] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:18:33.242667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.320585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4992e2a-dd77-44e6-a56d-670c47c391fd admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.325989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.326649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.326931 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.327259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.327940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.329895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.358402 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1273/2535] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:18:33 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:18:33.359956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.374105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a294cb-748a-467c-84c7-3fa0f3172339 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.433944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.434531 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.434816 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.435096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.435780 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.437731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.459952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1263/2536] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:18:33 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:33.463675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.473671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd37f8-6fee-4f5b-859e-8d85d4e113e5 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.480026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.480885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.481359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.481636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.482568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.484319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.498893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1274/2537] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:33.500340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.510676 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca070cbc-d715-4b79-98c2-4403c48bf827 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.515375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.516382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.516697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.516983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] PATH_INFO: `/v3/projects/d0e3b7e9d75e4dd5b0b74c05a4ba0d92/users/edc337aa781a42fb9f8e2ca684bce144/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.531379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e3b7e9d75e4dd5b0b74c05a4ba0d92, user_id=edc337aa781a42fb9f8e2ca684bce144, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.534058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.547547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1264/2538] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:33 2022] PUT /identity/v3/projects/d0e3b7e9d75e4dd5b0b74c05a4ba0d92/users/edc337aa781a42fb9f8e2ca684bce144/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:33.552491 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.563250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9e299-ef6d-44ad-903f-398fcde10860 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.620944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1125-45f5-4f18-bac2-563c4d11c8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.621612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1125-45f5-4f18-bac2-563c4d11c8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.621890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1125-45f5-4f18-bac2-563c4d11c8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.622158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1125-45f5-4f18-bac2-563c4d11c8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.622870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca1125-45f5-4f18-bac2-563c4d11c8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.624439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca1125-45f5-4f18-bac2-563c4d11c8b9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.637867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1275/2539] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:33.639081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.649956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b659995-41dc-4c2a-82bb-64d9a3929164 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.654569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.655466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.655747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.656017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] PATH_INFO: `/v3/projects/d0e3b7e9d75e4dd5b0b74c05a4ba0d92/users/edc337aa781a42fb9f8e2ca684bce144/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.658097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e3b7e9d75e4dd5b0b74c05a4ba0d92, user_id=edc337aa781a42fb9f8e2ca684bce144, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.660776 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.674138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1265/2540] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:33 2022] PUT /identity/v3/projects/d0e3b7e9d75e4dd5b0b74c05a4ba0d92/users/edc337aa781a42fb9f8e2ca684bce144/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:33.684904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.685555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.685555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.715393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] MFA Rules not processed for user `edc337aa781a42fb9f8e2ca684bce144`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:33.716856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:33.784779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1276/2541] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:33 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:33.795441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.797297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:33.853632 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac4de5-02b3-4031-8220-703415579712 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.857270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.857908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.858184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.858455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.868173 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.869565 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:33.879107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1266/2542] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:33.923982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:33.934091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3554e64-5e2d-4b9f-a922-42f319e3bb40 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:33.937622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:33.938248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:33.938524 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:33.938793 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:33.995160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:33.996585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:34.005323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1277/2543] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:36.795905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:36.797510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:36.867858 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff63f-2031-4dad-8a6f-448a86674920 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:36.873349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:36.874245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:36.874521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:36.874791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] PATH_INFO: `/v3/users/e5666513102246cfa1f033069fa8ecbd` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:36.875975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5666513102246cfa1f033069fa8ecbd)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:36.877917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:36.944096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5666513102246cfa1f033069fa8ecbd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:36.951814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5666513102246cfa1f033069fa8ecbd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:36.952670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5666513102246cfa1f033069fa8ecbd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:36.964287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5666513102246cfa1f033069fa8ecbd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:36.965096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5666513102246cfa1f033069fa8ecbd'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:36.985400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1267/2544] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:36 2022] DELETE /identity/v3/users/e5666513102246cfa1f033069fa8ecbd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:37.181809 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:37.207650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b211-a91d-48d2-bba4-045695369959 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:37.293039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:37.294023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:37.294369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:37.294641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] PATH_INFO: `/v3/projects/3efbfa7562e44ff686595413de7e3057` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:37.306344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efbfa7562e44ff686595413de7e3057)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:37.308989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:37.381464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efbfa7562e44ff686595413de7e3057', 'request_id': 'req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.394430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efbfa7562e44ff686595413de7e3057', 'request_id': 'req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.403059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efbfa7562e44ff686595413de7e3057', 'request_id': 'req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.415332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] The token cache is being invalidate because project 3efbfa7562e44ff686595413de7e3057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:37.415613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.416489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.418253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1278/2545] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:37 2022] DELETE /identity/v3/projects/3efbfa7562e44ff686595413de7e3057 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:37.421922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:37.423414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:37.436150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ab1a-9e3b-4da0-86b7-1f87f02ee3b7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:37.462962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:37.464654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:37.496054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:37.496959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:37.497240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:37.497512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] PATH_INFO: `/v3/users/c14a0556f40d409bb53542a418ac28d8` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:37.510711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14a0556f40d409bb53542a418ac28d8)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:37.512625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:37.526210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4827af1-1016-4ba9-8f65-f9cecc88fbd3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:37.531185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:37.532049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:37.532325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:37.532593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] PATH_INFO: `/v3/users/828ecca046c0412aacf39d4d94b900ad` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:37.533785 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828ecca046c0412aacf39d4d94b900ad)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:37.535938 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:37.556660 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14a0556f40d409bb53542a418ac28d8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.563938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14a0556f40d409bb53542a418ac28d8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.564624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14a0556f40d409bb53542a418ac28d8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.573080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14a0556f40d409bb53542a418ac28d8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.573684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-86b4c0fb-edd0-441f-9ee4-897f6e2f583e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14a0556f40d409bb53542a418ac28d8'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.580237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828ecca046c0412aacf39d4d94b900ad'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.580923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828ecca046c0412aacf39d4d94b900ad'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.588089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828ecca046c0412aacf39d4d94b900ad'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.591191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1268/2546] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:37 2022] DELETE /identity/v3/users/c14a0556f40d409bb53542a418ac28d8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:37.596551 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828ecca046c0412aacf39d4d94b900ad'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.597218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828ecca046c0412aacf39d4d94b900ad'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.614424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1279/2547] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:37 2022] DELETE /identity/v3/users/828ecca046c0412aacf39d4d94b900ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:37.619111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:37.619340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:37.619644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:37.637435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:37.638617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:37.703506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1269/2548] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:37.713851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:37.715240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:37.765036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:37.766428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:37.774533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac3945-738a-47da-b75f-ff4a421526c4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:37.778969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:37.779613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:37.779895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:37.780169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:37.790765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:37.792738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:37.794289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c6c63-4cc1-448e-b060-1dde810e2603 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:37.799480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:37.800357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:37.800794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:37.801101 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] PATH_INFO: `/v3/projects/25bd40642a5b499f810f9d32d44a4cf2` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:37.803217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1280/2549] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:37.813260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25bd40642a5b499f810f9d32d44a4cf2)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:37.815206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:37.869195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25bd40642a5b499f810f9d32d44a4cf2', 'request_id': 'req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.877144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25bd40642a5b499f810f9d32d44a4cf2', 'request_id': 'req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.886105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25bd40642a5b499f810f9d32d44a4cf2', 'request_id': 'req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.898254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] The token cache is being invalidate because project 25bd40642a5b499f810f9d32d44a4cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:37.898540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.899277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:37.901139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1270/2550] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:37 2022] DELETE /identity/v3/projects/25bd40642a5b499f810f9d32d44a4cf2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:18:38.113760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:38.115873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:38.137061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a0343-4240-4963-9242-48fedaef541d service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:38.199376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:38.200212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:38.200493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:38.200788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] PATH_INFO: `/v3/projects/6d2ee49f8ed843afbdb544ecb7b8fc7e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:38.201934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2ee49f8ed843afbdb544ecb7b8fc7e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:38.204049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:38.249625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2ee49f8ed843afbdb544ecb7b8fc7e', 'request_id': 'req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.255710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2ee49f8ed843afbdb544ecb7b8fc7e', 'request_id': 'req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.262817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2ee49f8ed843afbdb544ecb7b8fc7e', 'request_id': 'req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.273879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] The token cache is being invalidate because project 6d2ee49f8ed843afbdb544ecb7b8fc7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:38.274161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.274780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.276289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1281/2551] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:38 2022] DELETE /identity/v3/projects/6d2ee49f8ed843afbdb544ecb7b8fc7e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:38.280349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:38.281795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:38.295103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83567a8-7fa8-4e2a-a3f7-98eea724cd2d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:38.349160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:38.350016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:38.350292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:38.350561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] PATH_INFO: `/v3/users/edc337aa781a42fb9f8e2ca684bce144` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:38.351749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc337aa781a42fb9f8e2ca684bce144)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:38.353683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:38.397250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc337aa781a42fb9f8e2ca684bce144'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.403827 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc337aa781a42fb9f8e2ca684bce144'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.404803 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc337aa781a42fb9f8e2ca684bce144'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.412043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc337aa781a42fb9f8e2ca684bce144'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.412866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6d8358ce-4312-4871-8366-0f6651001b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc337aa781a42fb9f8e2ca684bce144'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.430692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1271/2552] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:38 2022] DELETE /identity/v3/users/edc337aa781a42fb9f8e2ca684bce144 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:38.695949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:38.697418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:38.712543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7586c-7b61-4c3e-9e55-21ddfeac0de6 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:38.769768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:38.770645 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:38.770928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:38.771203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] PATH_INFO: `/v3/projects/d0e3b7e9d75e4dd5b0b74c05a4ba0d92` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:38.772366 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e3b7e9d75e4dd5b0b74c05a4ba0d92)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:38.774300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:38.821742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e3b7e9d75e4dd5b0b74c05a4ba0d92', 'request_id': 'req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.828782 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e3b7e9d75e4dd5b0b74c05a4ba0d92', 'request_id': 'req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.836020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e3b7e9d75e4dd5b0b74c05a4ba0d92', 'request_id': 'req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.847527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] The token cache is being invalidate because project d0e3b7e9d75e4dd5b0b74c05a4ba0d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:38.847814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.848463 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-ee6b94ef-92cc-49c5-bf40-7fed45beecc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:38.850020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1282/2553] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:38 2022] DELETE /identity/v3/projects/d0e3b7e9d75e4dd5b0b74c05a4ba0d92 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:18:38.855258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:38.855804 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:38.855912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:38.878634 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:38.879818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:38.941373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1272/2554] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:38.948960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:38.949272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:38.949576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:38.967767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:38.968991 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:38.979784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1283/2555] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:38.981764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:38.993235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182199a3-b63b-4c1e-8383-aee8eed271de None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:38.998866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:38.999533 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:38.999992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.000113 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.000830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:18:39.001138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.002745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.014613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1273/2556] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:18:39.019133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.028502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d6982-25dc-4474-a073-2f42c1cae49a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.033290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.034040 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.034320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.034590 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.035282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.037710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.062746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1284/2557] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:18:39 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:39.066826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.076528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d79643-488f-47b6-a595-194d5d6688c1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.129920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.130577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.130859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.131136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.131818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.133759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.153184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1274/2558] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:18:39 2022] POST /identity/v3/users => generated 292 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:39.157237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.167965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f53de-fa81-4e09-9699-80064b4f72fb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.172408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.173079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.173356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.173627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.174332 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.175935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.186384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1285/2559] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:39.190086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.199959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca6a6-1cee-49a2-b502-1fe4d750475c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.204540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6694c90-eb9f-4e5f-b00a-3ec22f70e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.205210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694c90-eb9f-4e5f-b00a-3ec22f70e352 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.205491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694c90-eb9f-4e5f-b00a-3ec22f70e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.205769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694c90-eb9f-4e5f-b00a-3ec22f70e352 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.206484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6694c90-eb9f-4e5f-b00a-3ec22f70e352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.208097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6694c90-eb9f-4e5f-b00a-3ec22f70e352 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.217783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1275/2560] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:39.222991 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.233058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3f2d0-c661-4566-807c-ff426b18b9a3 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.237579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.238431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.238706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.238978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] PATH_INFO: `/v3/projects/15b4dffb7fa442e1b61357475853cd87/users/ea0b88004bc5471aa5dc3a9b23aefb65/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.253755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b4dffb7fa442e1b61357475853cd87, user_id=ea0b88004bc5471aa5dc3a9b23aefb65, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.256340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.268746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1286/2561] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:39 2022] PUT /identity/v3/projects/15b4dffb7fa442e1b61357475853cd87/users/ea0b88004bc5471aa5dc3a9b23aefb65/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:18:39.279425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.279716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.280002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.306107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] MFA Rules not processed for user `ea0b88004bc5471aa5dc3a9b23aefb65`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:39.307271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:39.374045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1276/2562] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:39.378832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.436899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b54ee4-ae55-4ccd-b7de-299985186294 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.441470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.442107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.442452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.442796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.443476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.445384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.469603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1287/2563] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:18:39 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:18:39.474211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.484414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc019c0f-2b0d-4306-9f85-68e2a2458915 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.537033 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.537705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.537982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.538252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.538947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.540897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.560947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1277/2564] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:18:39 2022] POST /identity/v3/users => generated 296 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:39.564852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.574865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218272ed-4e38-481b-a357-89928544f53f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.579853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.580526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.580847 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.581120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.581834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.583913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.594323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1288/2565] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:39.598502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.609211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a547c7-9c44-40d6-911c-e01e98420efc admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.613731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.614629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.614907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.615176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] PATH_INFO: `/v3/projects/dc171cb68db04b1d8713b8c2f821414c/users/a3c713c9877f4f7bbfb6a7a3cce0c1cf/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.629798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc171cb68db04b1d8713b8c2f821414c, user_id=a3c713c9877f4f7bbfb6a7a3cce0c1cf, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.632425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.646097 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1278/2566] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:39 2022] PUT /identity/v3/projects/dc171cb68db04b1d8713b8c2f821414c/users/a3c713c9877f4f7bbfb6a7a3cce0c1cf/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:18:39.650148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.659996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c32f4f-ff41-4d69-b144-591e02f06d2c admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.716560 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2343e85-9bd3-48a5-9f27-f031c110acf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.717253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2343e85-9bd3-48a5-9f27-f031c110acf0 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.717537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2343e85-9bd3-48a5-9f27-f031c110acf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.717814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2343e85-9bd3-48a5-9f27-f031c110acf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.718540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2343e85-9bd3-48a5-9f27-f031c110acf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.720218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2343e85-9bd3-48a5-9f27-f031c110acf0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.730135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1289/2567] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:39.734098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.744129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7944a-80f5-4737-bb89-c6ef8a6bb01c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.748810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.749705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.749985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.750260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] PATH_INFO: `/v3/projects/dc171cb68db04b1d8713b8c2f821414c/users/a3c713c9877f4f7bbfb6a7a3cce0c1cf/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.752200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc171cb68db04b1d8713b8c2f821414c, user_id=a3c713c9877f4f7bbfb6a7a3cce0c1cf, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.755383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.765223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1279/2568] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:39 2022] PUT /identity/v3/projects/dc171cb68db04b1d8713b8c2f821414c/users/a3c713c9877f4f7bbfb6a7a3cce0c1cf/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:39.775745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.776039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.776320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.801937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] MFA Rules not processed for user `a3c713c9877f4f7bbfb6a7a3cce0c1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:39.803120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:39.867568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1290/2569] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:39.879243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:39.881623 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:39.942960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24633cf5-15e9-42c7-8e79-bbd24c1c775b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:39.946834 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:39.947471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:39.947754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:39.948029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:39.956750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:39.959040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:39.968946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1280/2570] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:40.102403 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:40.111685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac285a5-3376-4221-be57-5a7c2995847d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:40.115711 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3749c6-1aae-4abb-925e-9e77b73623a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:40.116333 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3749c6-1aae-4abb-925e-9e77b73623a7 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:40.116612 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3749c6-1aae-4abb-925e-9e77b73623a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:40.116921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3749c6-1aae-4abb-925e-9e77b73623a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:40.173053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3749c6-1aae-4abb-925e-9e77b73623a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:40.174517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3749c6-1aae-4abb-925e-9e77b73623a7 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:40.183698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1291/2571] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:40.423811 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:40.482830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828b031-bdc3-4861-af12-bd44115d7aa8 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:40.487255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:40.488104 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:40.488384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:40.488689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] PATH_INFO: `/v3/users/ea0b88004bc5471aa5dc3a9b23aefb65` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:40.490425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0b88004bc5471aa5dc3a9b23aefb65)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:40.492327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:40.534854 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0b88004bc5471aa5dc3a9b23aefb65'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:40.542196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0b88004bc5471aa5dc3a9b23aefb65'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:40.542825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0b88004bc5471aa5dc3a9b23aefb65'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:40.549068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0b88004bc5471aa5dc3a9b23aefb65'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:40.549777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0b88004bc5471aa5dc3a9b23aefb65'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:40.566574 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1281/2572] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:40 2022] DELETE /identity/v3/users/ea0b88004bc5471aa5dc3a9b23aefb65 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:40.570990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:40.571498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:40.571601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:40.587957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:40.589180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:40.645027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1292/2573] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:40.654464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:40.655840 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:40.710845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6878d86-89db-4a2c-a098-685644c1f668 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:40.714454 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:40.715077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:40.715355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:40.715624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:40.724962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:40.726386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:40.735894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1282/2574] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:41.002102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.003580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:41.017605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a165f-1d83-4fd8-b60d-cee888b111d5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.022825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.023704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.023965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.024234 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] PATH_INFO: `/v3/projects/15b4dffb7fa442e1b61357475853cd87` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.025549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b4dffb7fa442e1b61357475853cd87)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.027462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.073055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b4dffb7fa442e1b61357475853cd87', 'request_id': 'req-2306cb3d-e896-4efb-a2a8-f761a70bdf33'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.078360 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b4dffb7fa442e1b61357475853cd87', 'request_id': 'req-2306cb3d-e896-4efb-a2a8-f761a70bdf33'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.086873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b4dffb7fa442e1b61357475853cd87', 'request_id': 'req-2306cb3d-e896-4efb-a2a8-f761a70bdf33'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.098296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] The token cache is being invalidate because project 15b4dffb7fa442e1b61357475853cd87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:41.099396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.099887 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.101529 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1293/2575] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:41 2022] DELETE /identity/v3/projects/15b4dffb7fa442e1b61357475853cd87 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:41.105610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.116926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9d308-ee8f-4a22-96fa-8e7a991bff4d service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.174653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.175613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.175960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.176321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] PATH_INFO: `/v3/users/a3c713c9877f4f7bbfb6a7a3cce0c1cf` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.177616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c713c9877f4f7bbfb6a7a3cce0c1cf)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.179585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.227059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c713c9877f4f7bbfb6a7a3cce0c1cf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.234199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c713c9877f4f7bbfb6a7a3cce0c1cf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.234874 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c713c9877f4f7bbfb6a7a3cce0c1cf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.242132 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c713c9877f4f7bbfb6a7a3cce0c1cf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.242759 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-60fe67bc-b21b-4fd8-8a06-113295a613c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c713c9877f4f7bbfb6a7a3cce0c1cf'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.260735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1283/2576] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:18:41 2022] DELETE /identity/v3/users/a3c713c9877f4f7bbfb6a7a3cce0c1cf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:41.516056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.517531 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:41.530827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cb3d-e896-4efb-a2a8-f761a70bdf33 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.586517 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.587356 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.587630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.587894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] PATH_INFO: `/v3/projects/dc171cb68db04b1d8713b8c2f821414c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.589090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc171cb68db04b1d8713b8c2f821414c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.590963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.634132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc171cb68db04b1d8713b8c2f821414c', 'request_id': 'req-710603d4-a738-440e-9cb3-7f11edfc904f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.639184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc171cb68db04b1d8713b8c2f821414c', 'request_id': 'req-710603d4-a738-440e-9cb3-7f11edfc904f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.645971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc171cb68db04b1d8713b8c2f821414c', 'request_id': 'req-710603d4-a738-440e-9cb3-7f11edfc904f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.656496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] The token cache is being invalidate because project dc171cb68db04b1d8713b8c2f821414c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:18:41.656807 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.657420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-710603d4-a738-440e-9cb3-7f11edfc904f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:18:41.658916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1294/2577] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:18:41 2022] DELETE /identity/v3/projects/dc171cb68db04b1d8713b8c2f821414c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:41.664043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.664361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.664657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.683000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:41.684181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:41.738722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1284/2578] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:41.747870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.748154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.748428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.766550 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:41.767713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:41.775338 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1295/2579] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:18:41.779329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.788798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ac0eb-1510-448d-91db-aa38c2a2b01a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.793728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.794425 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.794682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.794975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.795686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:18:41.795986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.797603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.809339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1285/2580] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:18:41.813383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.822735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826fe88-f7ec-4b19-8a31-d70c0584693d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.827193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.827816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.828093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.828361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.829083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.830935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.857796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1296/2581] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:18:41 2022] POST /identity/v3/projects => generated 369 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:18:41.858845 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.869262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a246b-f8e6-4e66-a025-3ba08a3231ef admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.924412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.925234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.925511 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.925783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.926490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.928993 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.951289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1286/2582] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:18:41 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:41.952476 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.963626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fe216-0035-42d1-b631-68d442e31452 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:41.968387 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:41.969208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:41.969539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:41.969816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:41.970528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:41.972103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:41.983319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1297/2583] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:41.987109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:41.997879 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c7ec7-ee2e-4c2f-aa35-78247b4c9176 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:42.002524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e990530-7345-4609-8168-65c724a56aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:42.003176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990530-7345-4609-8168-65c724a56aff admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:42.003655 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990530-7345-4609-8168-65c724a56aff admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:42.003942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990530-7345-4609-8168-65c724a56aff admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:42.004703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e990530-7345-4609-8168-65c724a56aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:42.006297 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e990530-7345-4609-8168-65c724a56aff admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:42.018077 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1287/2584] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:18:42.022007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:42.031555 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde205d2-3232-41e7-baff-6253b70ff8ca admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:42.036610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:42.037510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:42.037789 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:42.038061 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] PATH_INFO: `/v3/projects/6df2f3dc9adc4d1d958d5090099cce18/users/f08318b743404dc3b43631ac5b2fa266/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:42.053536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df2f3dc9adc4d1d958d5090099cce18, user_id=f08318b743404dc3b43631ac5b2fa266, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:42.056207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:42.068949 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1298/2585] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:42 2022] PUT /identity/v3/projects/6df2f3dc9adc4d1d958d5090099cce18/users/f08318b743404dc3b43631ac5b2fa266/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:42.074088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36cc7-d1d6-43ae-a15b-23272b4a468d None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:42.074411 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36cc7-d1d6-43ae-a15b-23272b4a468d None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:42.074688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb36cc7-d1d6-43ae-a15b-23272b4a468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:42.091169 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-dcb36cc7-d1d6-43ae-a15b-23272b4a468d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:42.092340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-dcb36cc7-d1d6-43ae-a15b-23272b4a468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:42.149873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1288/2586] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:42.159399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:42.161751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:42.217062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d564a-56f6-4847-a6a5-ae574e4186c8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:42.220662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:42.221294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:42.221578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:42.221855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:42.231045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:42.232451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:42.242304 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1299/2587] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:18:42.837800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:42.838093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:42.838370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:42.865199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] MFA Rules not processed for user `f08318b743404dc3b43631ac5b2fa266`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:42.866410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:42.950499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1289/2588] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:42.951850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:42.963741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598541-9ce9-44ad-b007-2f5d0972635a service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:42.967845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:42.968591 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:42.968966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:42.969330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:42.970106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:42.972119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:42.999621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1300/2589] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:18:42 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:43.006250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:43.019224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcebe-c57f-468f-a971-6ae7b93a4595 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:43.084455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:43.085160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:43.085447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:43.085725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:43.086414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:43.088330 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:43.108718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1290/2590] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:18:43 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:18:43.112770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:43.125241 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658412ae-4455-473a-aff6-53971297d671 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:43.130603 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:43.131259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:43.131544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:43.131813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:43.132523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:43.134202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:43.148424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1301/2591] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:18:43.150646 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:43.161662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5370ee6-7806-44c4-a7a9-1b9d62d4f754 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:43.169318 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:43.171655 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:43.171949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:43.172268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] PATH_INFO: `/v3/projects/98b606d1a50e4f7d9f50641d99d71054/users/21635092dc3647068a1b7194f13bb633/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:43.198258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b606d1a50e4f7d9f50641d99d71054, user_id=21635092dc3647068a1b7194f13bb633, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:43.201253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:43.230774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1291/2592] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:43 2022] PUT /identity/v3/projects/98b606d1a50e4f7d9f50641d99d71054/users/21635092dc3647068a1b7194f13bb633/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:43.231507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:43.242102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccf9f9-589e-4c9c-b198-22596468ebd4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:43.296069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6f10e-fe50-4ef5-9808-bc9cd11f34b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:43.296764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6f10e-fe50-4ef5-9808-bc9cd11f34b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:43.297049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6f10e-fe50-4ef5-9808-bc9cd11f34b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:43.297324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6f10e-fe50-4ef5-9808-bc9cd11f34b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:43.298034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6f10e-fe50-4ef5-9808-bc9cd11f34b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:43.299602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6f10e-fe50-4ef5-9808-bc9cd11f34b1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:43.309746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1302/2593] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:18:43.313863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:43.324450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a34bc5-bb8a-4b6e-a393-c9703ee8d7cd admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:43.329195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:43.330062 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:43.330931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:43.331213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] PATH_INFO: `/v3/projects/98b606d1a50e4f7d9f50641d99d71054/users/21635092dc3647068a1b7194f13bb633/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:43.333281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b606d1a50e4f7d9f50641d99d71054, user_id=21635092dc3647068a1b7194f13bb633, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:43.335999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:43.346363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1292/2594] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:18:43 2022] PUT /identity/v3/projects/98b606d1a50e4f7d9f50641d99d71054/users/21635092dc3647068a1b7194f13bb633/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:18:43.928863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:43.929398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:43.929674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:43.956293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] MFA Rules not processed for user `21635092dc3647068a1b7194f13bb633`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:18:43.958777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:44.029784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1303/2595] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:18:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:18:44.041689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:18:44.043447 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:18:44.104034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b738b-d674-439b-aa4e-7931339b9d5c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:18:44.107606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:18:44.108247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:18:44.108526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:18:44.108828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:18:44.162156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:18:44.163576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:18:44.172497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1293/2596] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:18:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:19:26.294889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:19:26.296358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:19:26.365633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578087-af2b-47fd-b472-0aad9cb09fc0 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:19:26.370998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:19:26.371626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:19:26.371907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:19:26.372180 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:19:26.373923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:19:26.428312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:19:26.429916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:19:26.438414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1304/2597] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:19:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:21:18.495539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:21:18.496941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:21:18.563148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db628e4-3669-4c18-9e8b-66b11018a77c service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:21:18.567975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:21:18.568605 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:21:18.569396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:21:18.569670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:21:18.571369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:21:18.582129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:21:18.583678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:21:18.593635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1294/2598] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:21:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:22:30.549891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:22:30.551336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:22:30.565664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de693164-7f2d-488d-808f-c339a7678492 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:22:30.572697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:22:30.573372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:22:30.573694 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:22:30.574005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:22:30.575868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:22:30.590568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:22:30.592269 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:22:30.606539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1305/2599] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:22:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:23:28.508687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:23:28.568766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83969be-90b4-4e24-bb78-f4e5d7a69025 service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:23:28.573255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:23:28.573897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:23:28.574178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:23:28.575081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:23:28.576871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:23:28.589971 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:23:28.591440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:23:28.602065 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1295/2600] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:23:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:23:31.235063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:23:31.236589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:23:31.263460 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93c8c-d9a7-411c-8a63-c2135548c442 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:23:31.267535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:23:31.268301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:23:31.268610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:23:31.268942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:23:31.270972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:23:31.282163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:23:31.283757 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:23:31.292722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1306/2601] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:23:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:23:31.900722 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:23:31.910743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42f735-30ee-4525-8b25-7b8bdaf5932a service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:23:31.914886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:23:31.915899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:23:31.916216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:23:31.916542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:23:31.919157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:23:31.934986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:23:31.936649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:23:31.948955 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1296/2602] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:23:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:24:26.686107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:24:26.698647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e451b1-57dc-47f4-9cff-2cde3c3d2a1e service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:24:26.704181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:24:26.704835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:24:26.705117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:24:26.705391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:24:26.712895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:24:26.714478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:24:26.740817 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1307/2603] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:24:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:24:29.872331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:24:29.881437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b554c-f60a-4183-9fa3-228877033132 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:24:29.885490 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:24:29.886110 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:24:29.886387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:24:29.886658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:24:29.888333 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:24:29.966418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:24:29.967848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:24:29.985373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1297/2604] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:24:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:24:30.027085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:24:30.035984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1325e-1063-43b1-88af-694adedd176c service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:24:30.039503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:24:30.040425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:24:30.040742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:24:30.041019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:24:30.042758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:24:30.117721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:24:30.119553 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:24:30.137515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1308/2605] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:24:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:24:30.185466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:24:30.195254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe1d49-86ab-4778-a559-bac8cde27c74 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:24:30.199368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:24:30.199994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:24:30.200276 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:24:30.200576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:24:30.209042 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:24:30.210463 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:24:30.219175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1298/2606] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:24:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:26:21.507377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:26:21.508805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:26:21.581329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6290a-166e-4936-857e-38426b349a5e service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:26:21.586197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:26:21.586828 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:26:21.587103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:26:21.587372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:26:21.595481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:26:21.597069 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:26:21.607046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1309/2607] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:26:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:28:31.152162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:31.183488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59855304-108e-4d7f-b653-3bf59dd464ac service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:31.187365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:31.188093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:31.188441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:31.188750 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:31.190613 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:28:31.278000 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:31.279706 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:31.301329 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1299/2608] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:28:34.254734 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:34.276940 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc19535-0386-4e39-9a02-661fe36a8325 service placement] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:34.280577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:34.281309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:34.281588 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:34.281863 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:34.291373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:34.292865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:34.301169 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1310/2609] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:28:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:28:34.356567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:34.358072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:28:34.372309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceef270-b4da-4faa-b776-4b716d259891 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:34.377274 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:34.378120 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:34.378400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:34.378799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] PATH_INFO: `/v3/users/8fec0b9b99934e90b5c98eb43ca4a448` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:34.380037 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fec0b9b99934e90b5c98eb43ca4a448)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:34.382483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:34.441949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fec0b9b99934e90b5c98eb43ca4a448'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.450310 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fec0b9b99934e90b5c98eb43ca4a448'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.451043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fec0b9b99934e90b5c98eb43ca4a448'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.461073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fec0b9b99934e90b5c98eb43ca4a448'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.461767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-76763ec5-6091-4884-8754-32eae2ec15e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fec0b9b99934e90b5c98eb43ca4a448'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.484653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1300/2610] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:28:34 2022] DELETE /identity/v3/users/8fec0b9b99934e90b5c98eb43ca4a448 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:28:34.721964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:34.734130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87a933-4b08-477d-afdb-8ce16c03bda3 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:34.799953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:34.800844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:34.801122 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:34.801388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] PATH_INFO: `/v3/projects/004ab055d0c84fa08d1d238a46c04a3f` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:34.802580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004ab055d0c84fa08d1d238a46c04a3f)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:34.804446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:34.857364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004ab055d0c84fa08d1d238a46c04a3f', 'request_id': 'req-d82ae4b4-234f-43dd-b20b-1726268912f8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.862756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004ab055d0c84fa08d1d238a46c04a3f', 'request_id': 'req-d82ae4b4-234f-43dd-b20b-1726268912f8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.871739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004ab055d0c84fa08d1d238a46c04a3f', 'request_id': 'req-d82ae4b4-234f-43dd-b20b-1726268912f8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.883009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] The token cache is being invalidate because project 004ab055d0c84fa08d1d238a46c04a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:28:34.883321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.884541 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-d82ae4b4-234f-43dd-b20b-1726268912f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:28:34.886197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1311/2611] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:28:34 2022] DELETE /identity/v3/projects/004ab055d0c84fa08d1d238a46c04a3f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:28:34.892194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:34.892495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:34.892794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:34.947368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:28:34.948468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:28:35.005875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1301/2612] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:28:35.014450 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.014745 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.015027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.033644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:28:35.034843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:28:35.044030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1312/2613] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:28:35.048177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.057714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9d3fd-eb7c-4cd0-8611-f134e1714cf6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.062231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.062851 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.063138 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.063413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.064127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:28:35.064435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.066052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.077625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1302/2614] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:28:35.081714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.095989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c5db5-4886-48fe-bbc4-ba9b8073c333 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.100979 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.101638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.101920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.102192 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.102895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.104822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.147163 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1313/2615] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:28:35 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:28:35.148779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.159189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f6c29-6e00-4a76-9f29-d527707229e8 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.213651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.214215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.214494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.214799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.215479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.217414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.235976 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1303/2616] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:28:35 2022] POST /identity/v3/users => generated 291 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:28:35.239948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.249843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94268c-34f8-45da-91ae-2021789dc9df admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.254419 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.255064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.255621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.255621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.256313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.257929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.268145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1314/2617] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:28:35.271792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.281927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e926a-418c-4301-ba19-cf6daebbe555 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.286838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb0d93-0558-4a6f-a4f0-3a3a09f83565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.287478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb0d93-0558-4a6f-a4f0-3a3a09f83565 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.287755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb0d93-0558-4a6f-a4f0-3a3a09f83565 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.288026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb0d93-0558-4a6f-a4f0-3a3a09f83565 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.288769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb0d93-0558-4a6f-a4f0-3a3a09f83565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.290606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb0d93-0558-4a6f-a4f0-3a3a09f83565 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.300414 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1304/2618] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:28:35.304348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.313762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f093391-1496-4a1e-a05d-7124847974da admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.318271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.319119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.319393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.319662 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] PATH_INFO: `/v3/projects/e060840464e541c6b3346b7ec71eab15/users/9e893d42e7874be0966d43ef7fb2aa77/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.335093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e060840464e541c6b3346b7ec71eab15, user_id=9e893d42e7874be0966d43ef7fb2aa77, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.338436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.351424 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1315/2619] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:28:35 2022] PUT /identity/v3/projects/e060840464e541c6b3346b7ec71eab15/users/9e893d42e7874be0966d43ef7fb2aa77/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:28:35.362236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.362790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.362790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.390673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] MFA Rules not processed for user `9e893d42e7874be0966d43ef7fb2aa77`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:28:35.391951 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:28:35.457123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1305/2620] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:28:35.468117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:28:35.469724 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:28:35.572559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426a194-2f56-4dfb-9d87-c58cf08fa2f1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:28:35.578155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:28:35.578795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:28:35.579131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:28:35.579429 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:28:35.587053 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:28:35.588610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:28:35.597065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1316/2621] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:28:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:29:31.137031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:31.138408 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:31.154011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251d4d-e1c3-4b24-bdf6-0fc8ea751a3c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:31.159529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:31.160166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:31.160444 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:31.160746 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:31.162503 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:31.218362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:31.219865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8935d-93d8-44c1-a3fa-3013bc614949 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:31.229251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1306/2622] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:29:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:29:35.534250 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:35.592822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe549-a53f-40e8-bd9c-103420d61392 service nova] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:35.597300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:35.598142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:35.598418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:35.598688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] PATH_INFO: `/v3/users/9e893d42e7874be0966d43ef7fb2aa77` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:35.599910 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e893d42e7874be0966d43ef7fb2aa77)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:35.601813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:35.645788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e893d42e7874be0966d43ef7fb2aa77'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:35.646422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e893d42e7874be0966d43ef7fb2aa77'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:35.652918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e893d42e7874be0966d43ef7fb2aa77'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:35.659027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e893d42e7874be0966d43ef7fb2aa77'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:35.659738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e893d42e7874be0966d43ef7fb2aa77'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:35.675966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1317/2623] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:29:35 2022] DELETE /identity/v3/users/9e893d42e7874be0966d43ef7fb2aa77 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:29:35.681001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:35.681491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:35.681606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:35.698873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:29:35.700090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:35.756377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1307/2624] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:29:35.766652 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:35.768024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:35.823213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b991-36d0-4726-a589-05a54e3701c5 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:35.826854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd8fd-8831-441d-9f40-1cd9ae7d68dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:35.827486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd8fd-8831-441d-9f40-1cd9ae7d68dd service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:35.827769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd8fd-8831-441d-9f40-1cd9ae7d68dd service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:35.828044 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd8fd-8831-441d-9f40-1cd9ae7d68dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:35.836820 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bd8fd-8831-441d-9f40-1cd9ae7d68dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:35.838293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bd8fd-8831-441d-9f40-1cd9ae7d68dd service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:35.847488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1318/2625] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:29:36.109651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.111070 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:36.122899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520b5d1-1009-4a75-bea8-1884121f1368 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.128323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.129589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.129871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.130147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] PATH_INFO: `/v3/projects/e060840464e541c6b3346b7ec71eab15` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.131357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e060840464e541c6b3346b7ec71eab15)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.134498 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.184175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e060840464e541c6b3346b7ec71eab15', 'request_id': 'req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:36.189285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e060840464e541c6b3346b7ec71eab15', 'request_id': 'req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:36.195250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e060840464e541c6b3346b7ec71eab15', 'request_id': 'req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:36.208117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] The token cache is being invalidate because project e060840464e541c6b3346b7ec71eab15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:29:36.208401 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:36.209053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-5ab54b9e-f301-4a63-b4c8-69b3427ecf5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:29:36.211126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1308/2626] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:29:36 2022] DELETE /identity/v3/projects/e060840464e541c6b3346b7ec71eab15 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:29:36.216108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.216398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.216696 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.234573 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:29:36.235769 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:36.293353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1319/2627] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:29:36.301238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.301528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.301802 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.322974 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:29:36.324165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:36.332309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1309/2628] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:29:36.338133 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.347872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd5675-fa3d-4a42-81db-a05a9141e9b8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.352654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.353313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.353596 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.353900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.354625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:29:36.354934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.356521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.370089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1320/2629] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:29:36.374478 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.384324 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb28990-9cfd-4cb4-9a67-b7eb480fee95 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.388883 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.389527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.389821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.390071 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.390748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.392717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.419935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1310/2630] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:29:36.424554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.434864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f9e5a-e801-4dcd-af23-b354d6aae2eb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.493944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.494816 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.495270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.495704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.496392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.498391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.522564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1321/2631] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:29:36.523910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.538099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed43816-124c-44d9-b86b-8e60949fbe17 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.543124 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.543909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.544195 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.544469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.545225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.546889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.557818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1311/2632] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:29:36.561882 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.572137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49e94d-7b2a-4df1-b72b-2fe5099bf1eb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.576818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa0a87-3f60-400d-a529-cf8088c53881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.577461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa0a87-3f60-400d-a529-cf8088c53881 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.577737 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa0a87-3f60-400d-a529-cf8088c53881 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.578005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa0a87-3f60-400d-a529-cf8088c53881 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.578710 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa0a87-3f60-400d-a529-cf8088c53881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.580277 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa0a87-3f60-400d-a529-cf8088c53881 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.590860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1322/2633] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:29:36.594910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.604701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294b522-957f-46f5-af96-fedf6e7f8fd5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.609273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.610134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.610412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.610678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] PATH_INFO: `/v3/projects/f3bda9226193448ca149be0c3fe3c043/users/c197e5529c8b45ffa1f89a314d74658c/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.624905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bda9226193448ca149be0c3fe3c043, user_id=c197e5529c8b45ffa1f89a314d74658c, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.627541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.644551 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1312/2634] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:29:36 2022] PUT /identity/v3/projects/f3bda9226193448ca149be0c3fe3c043/users/c197e5529c8b45ffa1f89a314d74658c/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:29:36.656561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.656864 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.657141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.682686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] MFA Rules not processed for user `c197e5529c8b45ffa1f89a314d74658c`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:29:36.683961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:36.749186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1323/2635] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:29:36.753483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.813417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998720-0c6f-4262-b59f-8436e88adcaa admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.818086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.819233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.819512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.819784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.820481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.822403 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.847475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1313/2636] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/projects => generated 397 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:29:36.851482 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.861721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aeade2-67ba-4df2-ac22-f2db1db65c8d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.915873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.916537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.916846 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.917176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.917859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.919760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.938772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1324/2637] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:29:36 2022] POST /identity/v3/users => generated 305 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:29:36.942577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.952426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06282e4b-e883-4eb8-97e3-d4e0493a733a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.957007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.957658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.957941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.958215 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:36.958934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:36.961370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:36.971517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1314/2638] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:29:36.976195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:36.988474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c48523-c245-4711-9dfa-cb5f05d2e75f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:36.994766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:36.995773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:36.996054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:36.996326 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] PATH_INFO: `/v3/projects/e280926909df4e4eb8c020de2ba07b11/users/f47daa3740fe492dabe4cc731fa60bfe/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:37.016398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e280926909df4e4eb8c020de2ba07b11, user_id=f47daa3740fe492dabe4cc731fa60bfe, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:37.018972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:37.031835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1325/2639] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:29:36 2022] PUT /identity/v3/projects/e280926909df4e4eb8c020de2ba07b11/users/f47daa3740fe492dabe4cc731fa60bfe/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:29:37.035986 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:37.046135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9653666-1530-40fd-bc94-cb8661301c1a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:37.099763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9ea6e-15c6-4478-be67-73b5e5c4c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:37.100422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9ea6e-15c6-4478-be67-73b5e5c4c718 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:37.100736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9ea6e-15c6-4478-be67-73b5e5c4c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:37.101020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9ea6e-15c6-4478-be67-73b5e5c4c718 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:37.101731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9ea6e-15c6-4478-be67-73b5e5c4c718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:37.103316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9ea6e-15c6-4478-be67-73b5e5c4c718 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:37.112841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1315/2640] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:29:37.117425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:29:37.127290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6043c-f9c1-49c3-98d3-5b2b1eb9c595 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:29:37.132349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:29:37.133246 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:37.133522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:37.133787 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] PATH_INFO: `/v3/projects/e280926909df4e4eb8c020de2ba07b11/users/f47daa3740fe492dabe4cc731fa60bfe/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:37.135693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e280926909df4e4eb8c020de2ba07b11, user_id=f47daa3740fe492dabe4cc731fa60bfe, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:29:37.138336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:29:37.148441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1326/2641] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:29:37 2022] PUT /identity/v3/projects/e280926909df4e4eb8c020de2ba07b11/users/f47daa3740fe492dabe4cc731fa60bfe/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:29:37.158244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:29:37.158530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:29:37.158806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:29:37.184469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] MFA Rules not processed for user `f47daa3740fe492dabe4cc731fa60bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:29:37.185674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:29:37.249643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1316/2642] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:30:31.131761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:31.133508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.166820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:31.168192 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.211849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f9c65-6ca7-42bc-a134-5a848b23faea admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:31.215321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:31.215954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:31.216231 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:31.216501 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:31.218287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.223449 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ad157-ad62-4656-aaa8-184d01423961 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:31.227052 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:31.227687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:31.227969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:31.228245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:31.229987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.294235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:31.295669 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:31.298079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:31.299833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:31.309798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1317/2643] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:30:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:30:31.316096 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1327/2644] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:30:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:30:31.353171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:31.361831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4f53-acf9-4c46-be73-690e1f9a7712 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:31.365546 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:31.366777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:31.367056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:31.367339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:31.369171 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.437311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:31.438748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:31.450379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1318/2645] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:30:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:30:31.730714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:31.745185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272e98f-0668-49a4-9225-01f690a49425 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:31.745185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:31.745185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:31.745185 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:31.745566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:31.759752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:31.761346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:31.772432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1328/2646] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:30:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:30:31.844837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:31.846507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.913679 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054831d-28f0-4c76-823c-97e68856a797 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:31.917319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:31.917956 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:31.918234 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:31.918506 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:31.920245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:31.980667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:31.982040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:31.991372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1319/2647] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:30:32.960186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:32.961634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:32.974756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6949d-46c8-49a3-a4dd-7be389b3a0c5 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:32.979727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:32.980578 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:32.980894 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:32.981165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] PATH_INFO: `/v3/users/f08318b743404dc3b43631ac5b2fa266` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:32.982323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08318b743404dc3b43631ac5b2fa266)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:32.984210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:33.026941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08318b743404dc3b43631ac5b2fa266'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.027584 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08318b743404dc3b43631ac5b2fa266'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.034225 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08318b743404dc3b43631ac5b2fa266'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.042017 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08318b743404dc3b43631ac5b2fa266'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.042687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08318b743404dc3b43631ac5b2fa266'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.059526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1329/2648] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:30:32 2022] DELETE /identity/v3/users/f08318b743404dc3b43631ac5b2fa266 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:30:33.232451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:33.242338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04148b5b-6d61-431b-9abf-8cc6f7b13aea service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:33.297180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:33.298034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:33.298314 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:33.298587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] PATH_INFO: `/v3/projects/6df2f3dc9adc4d1d958d5090099cce18` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:33.299783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df2f3dc9adc4d1d958d5090099cce18)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:33.301702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:33.345838 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df2f3dc9adc4d1d958d5090099cce18', 'request_id': 'req-a304ed2e-f334-4911-a80d-bc5156bdafed'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.351656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df2f3dc9adc4d1d958d5090099cce18', 'request_id': 'req-a304ed2e-f334-4911-a80d-bc5156bdafed'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.357517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df2f3dc9adc4d1d958d5090099cce18', 'request_id': 'req-a304ed2e-f334-4911-a80d-bc5156bdafed'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.368566 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] The token cache is being invalidate because project 6df2f3dc9adc4d1d958d5090099cce18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:30:33.368875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.369484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.370978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1320/2649] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:30:33 2022] DELETE /identity/v3/projects/6df2f3dc9adc4d1d958d5090099cce18 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:30:33.374919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:33.376335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:33.389301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138d586-7247-4709-960a-fc35011ed382 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:33.443190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:33.444655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:33.444947 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:33.445214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] PATH_INFO: `/v3/users/21635092dc3647068a1b7194f13bb633` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:33.457610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=21635092dc3647068a1b7194f13bb633)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:33.459526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:33.501658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21635092dc3647068a1b7194f13bb633'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.502282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21635092dc3647068a1b7194f13bb633'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.508824 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21635092dc3647068a1b7194f13bb633'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.515755 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21635092dc3647068a1b7194f13bb633'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.516422 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-14fd14ca-6fa3-4075-a401-afd116221cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21635092dc3647068a1b7194f13bb633'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.532680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1330/2650] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:30:33 2022] DELETE /identity/v3/users/21635092dc3647068a1b7194f13bb633 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:30:33.703319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:33.704814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:33.717448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304ed2e-f334-4911-a80d-bc5156bdafed admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:33.772745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:33.773657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:33.773962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:33.774268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] PATH_INFO: `/v3/projects/98b606d1a50e4f7d9f50641d99d71054` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:33.784256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b606d1a50e4f7d9f50641d99d71054)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:33.786187 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:33.829423 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b606d1a50e4f7d9f50641d99d71054', 'request_id': 'req-afe37c67-8cca-4211-94de-5fdf8f96f3a2'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.835075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b606d1a50e4f7d9f50641d99d71054', 'request_id': 'req-afe37c67-8cca-4211-94de-5fdf8f96f3a2'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.841309 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b606d1a50e4f7d9f50641d99d71054', 'request_id': 'req-afe37c67-8cca-4211-94de-5fdf8f96f3a2'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.853456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] The token cache is being invalidate because project 98b606d1a50e4f7d9f50641d99d71054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:30:33.853739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.854341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-afe37c67-8cca-4211-94de-5fdf8f96f3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:30:33.855863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1321/2651] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:30:33 2022] DELETE /identity/v3/projects/98b606d1a50e4f7d9f50641d99d71054 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:30:33.862118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:33.862401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:33.862675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:33.880224 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:30:33.881365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:33.937275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1331/2652] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:30:33.946287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:33.946575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:33.946855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:33.964381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:30:33.965610 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:33.973702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1322/2653] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:30:33.977831 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:33.987240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732a6f3-bcc9-4e30-99b8-b57cd9841f86 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:33.992226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:33.992892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:33.993171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:33.993440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:33.994146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:30:33.994447 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:33.996007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.007557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1332/2654] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:30:34.011786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:34.020821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5de32-3f47-490d-b6ac-2da0ae099e96 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:34.025479 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:34.026112 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.026392 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.026663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.027402 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:34.029332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.054796 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1323/2655] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:30:34 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:30:34.058900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:34.068538 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5494ae-bbea-4c1d-86c5-83fb23eb7bb4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:34.121525 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:34.122167 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.122446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.122717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.123390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:34.125310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.147739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1333/2656] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:30:34 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:30:34.152298 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:34.162441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047affb-93e3-4bf5-bc06-f0bc484f400a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:34.167476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:34.168134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.168415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.168720 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.169473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:34.171106 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.181354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1324/2657] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:30:34.185318 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:34.195919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf508-ddfb-40d3-ba67-82e114d6a505 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:34.200727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcf1bc-a86a-4dc1-944a-65b7c8143c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:34.201376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcf1bc-a86a-4dc1-944a-65b7c8143c33 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.201654 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcf1bc-a86a-4dc1-944a-65b7c8143c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.201925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcf1bc-a86a-4dc1-944a-65b7c8143c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.202639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcf1bc-a86a-4dc1-944a-65b7c8143c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:34.204197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcf1bc-a86a-4dc1-944a-65b7c8143c33 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.213929 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1334/2658] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:30:34.218073 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:34.228323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731ba39-ab65-4a84-a655-ad9ec43b6c03 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:34.233002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:34.233873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.234151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.234428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] PATH_INFO: `/v3/projects/31f671b0d6a84e17b523c4c379b462d9/users/554ec94b0144454a85b31da256af7f39/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.249091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f671b0d6a84e17b523c4c379b462d9, user_id=554ec94b0144454a85b31da256af7f39, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:34.251711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.264214 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1325/2659] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:30:34 2022] PUT /identity/v3/projects/31f671b0d6a84e17b523c4c379b462d9/users/554ec94b0144454a85b31da256af7f39/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:30:34.274291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.274577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.274849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.301007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] MFA Rules not processed for user `554ec94b0144454a85b31da256af7f39`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:30:34.302178 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:34.362469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1335/2660] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:30:34.374365 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:30:34.375997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:30:34.450886 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba3d94-ca1c-4d49-94ac-1c4021399daf admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:30:34.454453 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbdba20bd1c243248348d5c5672fefbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:30:34.455082 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:30:34.455364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:30:34.455639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:30:34.463642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:30:34.465092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:30:34.473662 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1326/2661] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:30:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:31:29.458627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:31:29.460008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:31:29.525861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eb203-72fa-4450-861a-bc1ce5616717 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:31:29.530778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:31:29.531408 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:31:29.531695 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:31:29.531971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:31:29.533732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:31:29.548046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:31:29.549644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf8570-bdf4-45a5-9b0c-5d274fb7a695 service placement] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:31:29.561566 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1336/2662] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:31:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:27.279262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:27.280940 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:27.353828 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7432f3f-51e1-4212-955e-35c875fea915 service glance] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:27.362246 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:27.363117 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:27.363480 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:27.363649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] PATH_INFO: `/v3/users/c197e5529c8b45ffa1f89a314d74658c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:27.364852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c197e5529c8b45ffa1f89a314d74658c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:27.366762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:27.422339 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c197e5529c8b45ffa1f89a314d74658c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:27.429568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c197e5529c8b45ffa1f89a314d74658c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:27.430645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c197e5529c8b45ffa1f89a314d74658c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:27.439777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c197e5529c8b45ffa1f89a314d74658c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:27.440388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c197e5529c8b45ffa1f89a314d74658c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:27.459935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1327/2663] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:27 2022] DELETE /identity/v3/users/c197e5529c8b45ffa1f89a314d74658c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:27.464602 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:27.465761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:27.465761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:27.484552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:27.485748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:27.542438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1337/2664] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:27.553197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:27.554661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:27.609269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c093392-5e41-49d3-b8a2-4cd786ace17e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:27.612848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:27.613473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:27.613749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:27.614020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:27.623127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:27.624537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:27.633729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1328/2665] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:27.925867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:27.927278 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:27.939732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138c3fd-5df9-45e1-9a67-ec5abb2e337f None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:27.945972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:27.947203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:27.947497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:27.947786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] PATH_INFO: `/v3/projects/f3bda9226193448ca149be0c3fe3c043` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:27.949009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bda9226193448ca149be0c3fe3c043)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:27.952150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.006908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bda9226193448ca149be0c3fe3c043', 'request_id': 'req-104d403f-72b1-4bef-8e09-078c51394736'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.012457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bda9226193448ca149be0c3fe3c043', 'request_id': 'req-104d403f-72b1-4bef-8e09-078c51394736'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.020007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bda9226193448ca149be0c3fe3c043', 'request_id': 'req-104d403f-72b1-4bef-8e09-078c51394736'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.031212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] The token cache is being invalidate because project f3bda9226193448ca149be0c3fe3c043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:28.031496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.032190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.033871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1338/2666] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:27 2022] DELETE /identity/v3/projects/f3bda9226193448ca149be0c3fe3c043 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:28.038569 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.048794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5c622-6f11-4416-ab76-6b0e1a70e5f6 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.108387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.109286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.109564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.109837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] PATH_INFO: `/v3/users/f47daa3740fe492dabe4cc731fa60bfe` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.111014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47daa3740fe492dabe4cc731fa60bfe)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:28.112949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.161932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47daa3740fe492dabe4cc731fa60bfe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.169871 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47daa3740fe492dabe4cc731fa60bfe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.170514 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47daa3740fe492dabe4cc731fa60bfe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.177400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47daa3740fe492dabe4cc731fa60bfe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.178108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-58e5e0ab-1cd4-4400-89de-3810200c2256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47daa3740fe492dabe4cc731fa60bfe'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.195813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1329/2667] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:28 2022] DELETE /identity/v3/users/f47daa3740fe492dabe4cc731fa60bfe => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:28.483354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.484943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:28.498045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d403f-72b1-4bef-8e09-078c51394736 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.558233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.559120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.559395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.559664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] PATH_INFO: `/v3/projects/e280926909df4e4eb8c020de2ba07b11` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.561021 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e280926909df4e4eb8c020de2ba07b11)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:28.563019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.606853 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e280926909df4e4eb8c020de2ba07b11', 'request_id': 'req-9dc88305-0d84-4c30-b089-ab2a4292288e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.612074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e280926909df4e4eb8c020de2ba07b11', 'request_id': 'req-9dc88305-0d84-4c30-b089-ab2a4292288e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.619443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e280926909df4e4eb8c020de2ba07b11', 'request_id': 'req-9dc88305-0d84-4c30-b089-ab2a4292288e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.631545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] The token cache is being invalidate because project e280926909df4e4eb8c020de2ba07b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:28.631832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.632921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9dc88305-0d84-4c30-b089-ab2a4292288e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:28.636145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1339/2668] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:28 2022] DELETE /identity/v3/projects/e280926909df4e4eb8c020de2ba07b11 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:28.644998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.644998 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.645769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.667053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:28.668935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:28.725123 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1330/2669] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:28.732856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.733138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.733409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.751712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:28.752914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:28.760788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1340/2670] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:28.765279 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.774712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a67aa6-63b9-4197-aad1-daa81a17c87f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.779387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.780035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.780325 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.780598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.781343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:32:28.781654 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:28.783253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.795859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1331/2671] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:32:28.800090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.810128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dfea6-d5bf-42be-914e-9b1f89d3e479 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.814721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.815355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.815630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.815901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.816657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:28.818518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.844505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1341/2672] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:28 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:28.849003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.861666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43c5d2-bb29-4d3b-ac3d-d7387bf91f19 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.919716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.920732 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.921014 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.921288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.921973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:28.923909 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.943260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1332/2673] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:28 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:28.947259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.957395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281dbff7-9452-464e-8fb0-292163728dc8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.961999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.962644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.962923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.963194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.963898 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:28.965532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:28.975679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1342/2674] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:28.979473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:28.990625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfcdbf-1691-4ec7-8326-9d35eb77b728 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:28.995456 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85e07-d002-4cb7-9f16-202668d26216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:28.996203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85e07-d002-4cb7-9f16-202668d26216 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:28.997136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85e07-d002-4cb7-9f16-202668d26216 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:28.997464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85e07-d002-4cb7-9f16-202668d26216 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:28.998260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f85e07-d002-4cb7-9f16-202668d26216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.000029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f85e07-d002-4cb7-9f16-202668d26216 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.010625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1333/2675] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:29.014899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.024544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b842-dfc7-4a79-a56d-505e2dbf6c43 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.029089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.030119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.030392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.030659 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] PATH_INFO: `/v3/projects/c997ccb1e3354305a2cf464ed4338589/users/49abe553dde84687a8086959a01d192f/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.049401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c997ccb1e3354305a2cf464ed4338589, user_id=49abe553dde84687a8086959a01d192f, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.052108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.066391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1343/2676] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:29 2022] PUT /identity/v3/projects/c997ccb1e3354305a2cf464ed4338589/users/49abe553dde84687a8086959a01d192f/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:29.077984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.078281 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.078564 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.105250 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] MFA Rules not processed for user `49abe553dde84687a8086959a01d192f`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:29.107056 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:29.178806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1334/2677] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:29.180214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.261452 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7817-7e1b-424a-9ee8-9b224933bb04 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.265996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.266649 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.266930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.267207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.267900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.269821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.295164 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1344/2678] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:29 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:29.299177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.310959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d76d1-ef2d-4428-a4ee-a50b0829b21a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.373031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.373702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.373984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.374258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.374937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.376855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.400346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1335/2679] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:29 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:29.405254 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.416104 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eb5af-5f7a-4c16-ad7a-2bf33aa86f74 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.421320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.421959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.422235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.422503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.423212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.425059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.435753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1345/2680] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:29.439728 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.453846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc3f37-a937-4ad1-a106-4485727ebc74 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.459142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.460034 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.460313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.460587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] PATH_INFO: `/v3/projects/17638c3ed4cb49e3a98db3396638272b/users/cdcb0a89ba744a42b406cea48852f30c/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.478847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17638c3ed4cb49e3a98db3396638272b, user_id=cdcb0a89ba744a42b406cea48852f30c, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.482131 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.498160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1336/2681] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:29 2022] PUT /identity/v3/projects/17638c3ed4cb49e3a98db3396638272b/users/cdcb0a89ba744a42b406cea48852f30c/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:29.502208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.514237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b140683-5753-4528-9bfc-ef9b3b116ecf admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.581685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb90e0-5d17-4807-8e0c-ab1334b63ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.582373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb90e0-5d17-4807-8e0c-ab1334b63ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.582635 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb90e0-5d17-4807-8e0c-ab1334b63ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.582966 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb90e0-5d17-4807-8e0c-ab1334b63ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.583959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb90e0-5d17-4807-8e0c-ab1334b63ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.586339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb90e0-5d17-4807-8e0c-ab1334b63ea2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.607126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1346/2682] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:29.607393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.617651 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6b340-6e00-46b5-8216-7d755fcc0e0f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.622323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.623193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.623470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.623741 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] PATH_INFO: `/v3/projects/17638c3ed4cb49e3a98db3396638272b/users/cdcb0a89ba744a42b406cea48852f30c/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.625729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17638c3ed4cb49e3a98db3396638272b, user_id=cdcb0a89ba744a42b406cea48852f30c, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.628331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.639426 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1337/2683] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:29 2022] PUT /identity/v3/projects/17638c3ed4cb49e3a98db3396638272b/users/cdcb0a89ba744a42b406cea48852f30c/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:29.649767 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.650054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.650331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.676805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] MFA Rules not processed for user `cdcb0a89ba744a42b406cea48852f30c`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:29.677999 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:29.746723 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1347/2684] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:29.757530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:29.759376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:29.822204 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e90e-36d4-4342-80b4-f175dbf6b3c6 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:29.826562 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e93dc-8586-40ec-bd11-d300fabe85f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:29.827217 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e93dc-8586-40ec-bd11-d300fabe85f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:29.827496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e93dc-8586-40ec-bd11-d300fabe85f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:29.827762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e93dc-8586-40ec-bd11-d300fabe85f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:29.836938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e93dc-8586-40ec-bd11-d300fabe85f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:29.838369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e93dc-8586-40ec-bd11-d300fabe85f2 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:29.848004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1338/2685] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:30.213904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:30.276680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920acb98-6c4f-4d59-868e-224dac131bf8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:30.281239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:30.282137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:30.282412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:30.282679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] PATH_INFO: `/v3/users/49abe553dde84687a8086959a01d192f` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:30.283838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] RBAC: Authorizing `identity:delete_user(user_id=49abe553dde84687a8086959a01d192f)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:30.285765 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:30.331197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49abe553dde84687a8086959a01d192f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.331771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49abe553dde84687a8086959a01d192f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.338265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49abe553dde84687a8086959a01d192f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.345268 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49abe553dde84687a8086959a01d192f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.345885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49abe553dde84687a8086959a01d192f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.363680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1348/2686] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:30 2022] DELETE /identity/v3/users/49abe553dde84687a8086959a01d192f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:30.368687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:30.368687 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:30.368996 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:30.385958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:30.387189 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:30.457814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1339/2687] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:30.467319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:30.468754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:30.523617 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c9da-f430-4b10-9dc9-8f4b6353214b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:30.527486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:30.528203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:30.528528 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:30.528834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:30.538681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:30.540098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:30.550956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1349/2688] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:30.877949 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:30.879485 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:30.892335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72e15e-225b-4706-984a-a149ff9c20f7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:30.903232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:30.903870 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:30.904216 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:30.904477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] PATH_INFO: `/v3/projects/c997ccb1e3354305a2cf464ed4338589` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:30.906327 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c997ccb1e3354305a2cf464ed4338589)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:30.908296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:30.980262 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c997ccb1e3354305a2cf464ed4338589', 'request_id': 'req-7b857831-00a0-459f-bb19-4b6a0fb5f984'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.986727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c997ccb1e3354305a2cf464ed4338589', 'request_id': 'req-7b857831-00a0-459f-bb19-4b6a0fb5f984'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:30.993285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c997ccb1e3354305a2cf464ed4338589', 'request_id': 'req-7b857831-00a0-459f-bb19-4b6a0fb5f984'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.005908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] The token cache is being invalidate because project c997ccb1e3354305a2cf464ed4338589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:31.006188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.006790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.008349 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1340/2689] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:30 2022] DELETE /identity/v3/projects/c997ccb1e3354305a2cf464ed4338589 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:31.012962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.023758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14368b4-6a69-4bd1-92e8-8427e3b612b2 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.078202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.079148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.079407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.079681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] PATH_INFO: `/v3/users/cdcb0a89ba744a42b406cea48852f30c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.080885 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdcb0a89ba744a42b406cea48852f30c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.082777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.127217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdcb0a89ba744a42b406cea48852f30c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.127877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdcb0a89ba744a42b406cea48852f30c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.134437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdcb0a89ba744a42b406cea48852f30c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.141117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdcb0a89ba744a42b406cea48852f30c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.141746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-136b8a8c-d59a-473e-9425-3cf92e98c410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdcb0a89ba744a42b406cea48852f30c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.160608 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1350/2690] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:31 2022] DELETE /identity/v3/users/cdcb0a89ba744a42b406cea48852f30c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:31.431847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.433376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:31.446606 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b857831-00a0-459f-bb19-4b6a0fb5f984 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.504537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.505576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.505861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.506134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] PATH_INFO: `/v3/projects/17638c3ed4cb49e3a98db3396638272b` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.507428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17638c3ed4cb49e3a98db3396638272b)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.510388 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.557233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17638c3ed4cb49e3a98db3396638272b', 'request_id': 'req-711b8e65-edcd-4a5c-b1df-e0023b56232b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.563059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17638c3ed4cb49e3a98db3396638272b', 'request_id': 'req-711b8e65-edcd-4a5c-b1df-e0023b56232b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.569054 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17638c3ed4cb49e3a98db3396638272b', 'request_id': 'req-711b8e65-edcd-4a5c-b1df-e0023b56232b'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.581769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] The token cache is being invalidate because project 17638c3ed4cb49e3a98db3396638272b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:31.582049 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.582737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-711b8e65-edcd-4a5c-b1df-e0023b56232b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:31.584368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1341/2691] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:31 2022] DELETE /identity/v3/projects/17638c3ed4cb49e3a98db3396638272b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:31.589397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.589888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.590003 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.607050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:31.608216 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:31.666015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1351/2692] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:31.674937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.675164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.675437 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.693736 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:31.694916 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:31.703254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1342/2693] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:31.707599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.717497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84431-1efd-4874-a6ce-b7c78f5f0bd8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.722032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.722722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.723001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.723270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.724249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:32:31.724563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.726193 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.737753 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1352/2694] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:32:31.742208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.752149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13073b98-d4e6-4275-9897-6d78ff6a0dda None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.757377 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.758142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.758450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.758763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.759590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.761899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.794142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1343/2695] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:31 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:31.798563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.809869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce0ce5-f83c-43cb-b2cd-5f805a46d5ad admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.864428 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.865111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.865393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.865667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.866389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.868311 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.890773 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1353/2696] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:31 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:31.895108 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.905334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78144d-2bae-455a-a2d3-6135c73ca121 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.910166 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.910819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.911102 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.911374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.912096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.914013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.924782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1344/2697] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:31.928740 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.939260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec7869-ebbe-4a4e-b38e-e33617197610 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.944996 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f445d-e9d8-47db-9d48-207fba95036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.945599 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f445d-e9d8-47db-9d48-207fba95036a admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.945879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f445d-e9d8-47db-9d48-207fba95036a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.946205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f445d-e9d8-47db-9d48-207fba95036a admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.946935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f445d-e9d8-47db-9d48-207fba95036a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.948544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f445d-e9d8-47db-9d48-207fba95036a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:31.959622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1354/2698] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:31.963895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:31.973810 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77598eca-58f1-479c-afc6-ffcc0e5e589e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:31.978519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:31.979472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:31.979749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:31.980026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] PATH_INFO: `/v3/projects/c1d19e0cb15b4432bcb73d1bdb59596c/users/16fb5d767a674a2588f87d580b0e317e/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:31.995684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d19e0cb15b4432bcb73d1bdb59596c, user_id=16fb5d767a674a2588f87d580b0e317e, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:31.998930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.012539 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1345/2699] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:31 2022] PUT /identity/v3/projects/c1d19e0cb15b4432bcb73d1bdb59596c/users/16fb5d767a674a2588f87d580b0e317e/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:32.023766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.024046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.024316 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.050330 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] MFA Rules not processed for user `16fb5d767a674a2588f87d580b0e317e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:32.051508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:32.115295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1355/2700] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:32.119978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.180675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc63028-b8ad-45c7-8d8c-87880fd51fca admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.185176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.185818 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.186100 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.186375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.187061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.189013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.215524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1346/2701] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:32 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:32.219572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.229969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35124d5e-ac7c-418d-9f05-92161f0d1fa2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.285420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.286090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.286376 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.286650 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.287329 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.289533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.309987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1356/2702] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:32 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:32.314161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.325427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a9fbb-693c-4f16-b884-7c829a93a7c3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.332859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.333573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.333855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.334134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.334862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.336468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.348337 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1347/2703] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:32:32.352174 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.362593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7122815-e48c-4275-82a9-18084bf09506 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.367144 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.368023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.368295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.368559 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] PATH_INFO: `/v3/projects/9c78a01c3f894cf8b56a7822ed8e16bf/users/ea288ce6ba9e47769a9eab660905b53c/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.386253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c78a01c3f894cf8b56a7822ed8e16bf, user_id=ea288ce6ba9e47769a9eab660905b53c, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.389264 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.403147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1357/2704] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:32 2022] PUT /identity/v3/projects/9c78a01c3f894cf8b56a7822ed8e16bf/users/ea288ce6ba9e47769a9eab660905b53c/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:32.407213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.417364 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ce76-6a36-44dd-9aeb-37b769a2e7fb admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.471563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90da8e-a29f-4693-b83a-6b8c6baea434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.472146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90da8e-a29f-4693-b83a-6b8c6baea434 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.472421 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90da8e-a29f-4693-b83a-6b8c6baea434 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.472738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90da8e-a29f-4693-b83a-6b8c6baea434 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.473451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90da8e-a29f-4693-b83a-6b8c6baea434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.475027 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90da8e-a29f-4693-b83a-6b8c6baea434 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.484491 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1348/2705] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:32.488613 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.498647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf6b69-55af-4481-9ba0-98822fac0239 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.503171 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.504033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.504308 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.504577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] PATH_INFO: `/v3/projects/9c78a01c3f894cf8b56a7822ed8e16bf/users/ea288ce6ba9e47769a9eab660905b53c/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.506554 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c78a01c3f894cf8b56a7822ed8e16bf, user_id=ea288ce6ba9e47769a9eab660905b53c, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.509172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.519258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1358/2706] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:32 2022] PUT /identity/v3/projects/9c78a01c3f894cf8b56a7822ed8e16bf/users/ea288ce6ba9e47769a9eab660905b53c/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:32.530212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.530550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.530876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.556694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] MFA Rules not processed for user `ea288ce6ba9e47769a9eab660905b53c`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:32.557890 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:32.623450 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1349/2707] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:32.633935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:32.635629 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:32.691651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed3ee8-8498-49e0-a1c1-97b505dc42f8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:32.695384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:32.696013 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:32.696292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:32.696565 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:32.749301 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:32.750642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:32.759101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1359/2708] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:40.055543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:40.057011 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:40.113771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f2e3-7e81-4f99-aa42-7f50978dae7a None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:40.117301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:40.117917 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:40.118196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:40.118469 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:40.120190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:40.185238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:40.186671 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a69a79-7c6a-4cb8-bfc6-fde39fb4f97e service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:40.195899 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1350/2709] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:40.613827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:40.615835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:40.631102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338167-1d12-40f6-97f1-9249759d411e service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:40.637207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:40.638922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:40.639486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:40.639930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] PATH_INFO: `/v3/users/554ec94b0144454a85b31da256af7f39` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:40.642860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=554ec94b0144454a85b31da256af7f39)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:40.644957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:40.695673 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554ec94b0144454a85b31da256af7f39'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:40.696971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554ec94b0144454a85b31da256af7f39'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:40.705191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554ec94b0144454a85b31da256af7f39'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:40.713866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554ec94b0144454a85b31da256af7f39'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:40.714741 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554ec94b0144454a85b31da256af7f39'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:40.734689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1360/2710] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:40 2022] DELETE /identity/v3/users/554ec94b0144454a85b31da256af7f39 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:40.736549 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:40.736936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:40.737284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:40.754412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:40.755731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:40.818115 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1351/2711] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:40.828851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:40.830594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:40.891028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c75d-f83e-4c3b-836b-5aa7898e2be9 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:40.894815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:40.895457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:40.895735 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:40.896005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:40.905276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:40.906849 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:40.922976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1361/2712] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:40.939705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:40.941703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:40.960878 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eee1e3-fcfe-40f9-b791-5c7c2ca11f0e None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:40.965935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:40.966806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:40.967084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:40.967350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] PATH_INFO: `/v3/users/9a7f9e2344fa4cfab4e6836d634865ad` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:40.969224 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7f9e2344fa4cfab4e6836d634865ad)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:40.972164 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.024957 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7f9e2344fa4cfab4e6836d634865ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.032381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7f9e2344fa4cfab4e6836d634865ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.033358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7f9e2344fa4cfab4e6836d634865ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.041230 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7f9e2344fa4cfab4e6836d634865ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.044151 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7f9e2344fa4cfab4e6836d634865ad'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.075928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1352/2713] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:40 2022] DELETE /identity/v3/users/9a7f9e2344fa4cfab4e6836d634865ad => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:41.194098 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.196105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.261618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62d53-36ac-49d3-b5f5-0850ba165a92 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.316907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.318558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.323783 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.324899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.325221 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.325657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.328821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.415673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa5699-651b-4f1e-8d8d-4071b064d154 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.418510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.420007 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.420683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.421608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.421950 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.422289 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] PATH_INFO: `/v3/projects/31f671b0d6a84e17b523c4c379b462d9` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.423513 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f671b0d6a84e17b523c4c379b462d9)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.425512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.429729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1362/2714] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:41.430513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.431754 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.444445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c7b4-bca7-436d-be2a-3b8e64bc4bac service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.449772 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.450609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.450893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.451165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] PATH_INFO: `/v3/projects/9be561020bdd441096d407f87ac63011` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.452325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be561020bdd441096d407f87ac63011)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.454249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.480920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f671b0d6a84e17b523c4c379b462d9', 'request_id': 'req-9755128f-6cde-4dff-a1b3-baf12191a1d0'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.487076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f671b0d6a84e17b523c4c379b462d9', 'request_id': 'req-9755128f-6cde-4dff-a1b3-baf12191a1d0'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.495458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f671b0d6a84e17b523c4c379b462d9', 'request_id': 'req-9755128f-6cde-4dff-a1b3-baf12191a1d0'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.509869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be561020bdd441096d407f87ac63011', 'request_id': 'req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.510008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] The token cache is being invalidate because project 31f671b0d6a84e17b523c4c379b462d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:41.510228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.510837 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-9755128f-6cde-4dff-a1b3-baf12191a1d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.513373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1353/2715] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:41 2022] DELETE /identity/v3/projects/31f671b0d6a84e17b523c4c379b462d9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:41.516693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be561020bdd441096d407f87ac63011', 'request_id': 'req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.517783 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27bc7b-d0ef-4696-b1ff-c3214b1a053f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.518061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27bc7b-d0ef-4696-b1ff-c3214b1a053f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.518331 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27bc7b-d0ef-4696-b1ff-c3214b1a053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.526950 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be561020bdd441096d407f87ac63011', 'request_id': 'req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.537128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-3f27bc7b-d0ef-4696-b1ff-c3214b1a053f None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:41.538568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3f27bc7b-d0ef-4696-b1ff-c3214b1a053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.539998 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] The token cache is being invalidate because project 9be561020bdd441096d407f87ac63011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:41.540395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.541364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-2dfe6ea4-0cdc-46c8-8cf2-1f7c769fceb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:41.544256 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1363/2716] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:41 2022] DELETE /identity/v3/projects/9be561020bdd441096d407f87ac63011 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:41.552838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44928ce-5767-47f9-9a4f-d742982bf05b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.552838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44928ce-5767-47f9-9a4f-d742982bf05b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.552838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44928ce-5767-47f9-9a4f-d742982bf05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.571626 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b44928ce-5767-47f9-9a4f-d742982bf05b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:41.572835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b44928ce-5767-47f9-9a4f-d742982bf05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.614010 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1354/2717] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:32:41.630635 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.630937 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.631188 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.639073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1364/2718] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:32:41.648132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.648421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.648729 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.652848 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:41.654111 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.663589 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1355/2719] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:41.668670 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.670960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:41.672152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:41.678629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cae889-c223-476d-a1d5-52648fc819f5 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.681317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1365/2720] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:32:41.687097 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.687521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.688285 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.688656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.688991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.689788 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:32:41.690093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.700079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.706217 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f25e0-7b36-4379-85af-d4542334f05d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.711539 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.712344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.712704 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.712970 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.713666 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:32:41.714165 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.715796 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.723231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1356/2721] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:32:41.727475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.736354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1366/2722] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:32:41.737359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.744393 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf83d67-b5c3-4cf0-88d8-222ce1941949 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.750618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd856-3b9d-458d-9097-0452c7c0a299 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.750930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.751703 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.751985 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.752264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.753021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.759019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.762321 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.763072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.763467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.763818 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.764580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.766752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.793774 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1357/2723] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:41.794791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.795961 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1367/2724] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:41.799354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.875627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bae2e-13ae-4140-aae8-d925de2a990a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.881617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.882294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.882406 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff25058-1a2f-4445-812b-ca401c55e8ed admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.882650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.882931 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.883913 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.886843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.889389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.890594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.892369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.892794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.893732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.896177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.914386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1358/2725] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:41.918576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.920527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1368/2726] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:41 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:32:41.924101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.928223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c73b4-4ea7-4ae2-91d1-7e5a07a7457b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.932932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.934091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.934370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.934638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10657ef4-df8f-4f19-bf00-4b8b946b89df admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.934769 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.935346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.936979 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.939343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.940033 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.940456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.940867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.941611 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.943170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.946648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1359/2727] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:32:41.951128 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.953840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1369/2728] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:41.957706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.962243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9fc8b-7314-4b90-85a1-2ed3ea6afac0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.966905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.967520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dcafc-f4d1-406c-b6cb-a075acb2f00d admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:41.967643 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.967824 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.968098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.968847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.970429 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.972340 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:41.973008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:41.973286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:41.973557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:41.974266 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:41.975832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:41.980865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1360/2729] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:32:41.986658 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.988521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1370/2730] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:32:41.992265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:41.999340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4fcc-aa14-48ec-adfa-40e745c8245f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:42.002319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4a780-3d17-4b28-8a06-1e1c577bd280 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:42.004396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba3868-cdc7-4843-a619-9886babd0cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:42.005752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba3868-cdc7-4843-a619-9886babd0cde admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.006277 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba3868-cdc7-4843-a619-9886babd0cde admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.006585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba3868-cdc7-4843-a619-9886babd0cde admin admin] PATH_INFO: `/v3/projects/768544c9d920467ea2cef339e3c496d0/users/eda254dbd85349ee97600e446807211e/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.007499 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3def470-7a6e-4fa5-a795-dd6cd91ab8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:42.008361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3def470-7a6e-4fa5-a795-dd6cd91ab8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.008651 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3def470-7a6e-4fa5-a795-dd6cd91ab8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.008932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3def470-7a6e-4fa5-a795-dd6cd91ab8d3 admin admin] PATH_INFO: `/v3/projects/1954e5ac9597438b97fd388aa735a467/users/17e016e1e171494c9932df04761fb738/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.024153 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba3868-cdc7-4843-a619-9886babd0cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=768544c9d920467ea2cef339e3c496d0, user_id=eda254dbd85349ee97600e446807211e, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:42.024380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3def470-7a6e-4fa5-a795-dd6cd91ab8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1954e5ac9597438b97fd388aa735a467, user_id=17e016e1e171494c9932df04761fb738, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:42.026920 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3def470-7a6e-4fa5-a795-dd6cd91ab8d3 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:42.027083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba3868-cdc7-4843-a619-9886babd0cde admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:42.041109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1371/2731] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:41 2022] PUT /identity/v3/projects/1954e5ac9597438b97fd388aa735a467/users/17e016e1e171494c9932df04761fb738/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:42.044773 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1361/2732] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:41 2022] PUT /identity/v3/projects/768544c9d920467ea2cef339e3c496d0/users/eda254dbd85349ee97600e446807211e/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:42.050290 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.050670 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.051015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.069964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:42.071190 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:42.102118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.102678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.102678 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.128473 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] MFA Rules not processed for user `eda254dbd85349ee97600e446807211e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:42.129977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:42.131915 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1372/2733] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:42.141454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:42.142786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:42.196249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1362/2734] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:32:42.200844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eef43-346f-4b95-a069-7b7790ffdb63 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:42.204443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:42.205118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.205404 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.205681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.207348 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:42.208695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:42.215718 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:42.217179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:42.226389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1373/2735] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:42.267096 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09bfff-4673-4e78-826f-cf459789a918 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:42.274619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423be9c6-d753-41ef-b291-3244d8b76732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:42.275307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423be9c6-d753-41ef-b291-3244d8b76732 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.275591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423be9c6-d753-41ef-b291-3244d8b76732 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.275859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423be9c6-d753-41ef-b291-3244d8b76732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.284932 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423be9c6-d753-41ef-b291-3244d8b76732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:42.286370 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423be9c6-d753-41ef-b291-3244d8b76732 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:42.296685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1363/2736] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:42.325249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:42.336821 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439a0dd-5675-4544-bb7d-23b17d7764de service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:42.341628 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:42.342620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.342899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.343172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] PATH_INFO: `/v3/users/eda254dbd85349ee97600e446807211e` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.344365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda254dbd85349ee97600e446807211e)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:42.346317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:42.397688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda254dbd85349ee97600e446807211e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:42.398384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda254dbd85349ee97600e446807211e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:42.407136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda254dbd85349ee97600e446807211e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:42.417232 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda254dbd85349ee97600e446807211e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:42.418386 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda254dbd85349ee97600e446807211e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:42.436129 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1374/2737] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:42 2022] DELETE /identity/v3/users/eda254dbd85349ee97600e446807211e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:42.441441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.441901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.442254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.459990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:42.461210 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:42.545499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1364/2738] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:42.555293 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:42.556721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:42.620102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2f8cb-cadb-4a21-befd-537256c7ec14 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:42.624839 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfeacf-b294-4d2a-a112-11673cdccaa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:42.626255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfeacf-b294-4d2a-a112-11673cdccaa9 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:42.626255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfeacf-b294-4d2a-a112-11673cdccaa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:42.626255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfeacf-b294-4d2a-a112-11673cdccaa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:42.637855 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfeacf-b294-4d2a-a112-11673cdccaa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:42.639545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfeacf-b294-4d2a-a112-11673cdccaa9 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:42.651060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1375/2739] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:43.016460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:43.017992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:43.037357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42479778-8407-4760-a4de-8fb521fbcba0 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:43.055545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:43.056434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:43.056892 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:43.057081 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] PATH_INFO: `/v3/projects/768544c9d920467ea2cef339e3c496d0` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:43.058813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=768544c9d920467ea2cef339e3c496d0)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:43.060794 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:43.126284 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768544c9d920467ea2cef339e3c496d0', 'request_id': 'req-bde0cba7-fb15-445a-9e04-108277488a24'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:43.132148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768544c9d920467ea2cef339e3c496d0', 'request_id': 'req-bde0cba7-fb15-445a-9e04-108277488a24'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:43.140013 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768544c9d920467ea2cef339e3c496d0', 'request_id': 'req-bde0cba7-fb15-445a-9e04-108277488a24'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:43.159075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] The token cache is being invalidate because project 768544c9d920467ea2cef339e3c496d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:43.159361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:43.160249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:43.184677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1365/2740] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:43 2022] DELETE /identity/v3/projects/768544c9d920467ea2cef339e3c496d0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:46.275214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.275759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.275759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.303669 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] MFA Rules not processed for user `17e016e1e171494c9932df04761fb738`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:46.304968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:46.375493 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1376/2741] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:46.380249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.381743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:46.446812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0cba7-fb15-445a-9e04-108277488a24 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.452484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.453193 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.453476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.453752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.454470 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.456379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.489412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1366/2742] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:46 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:46.490053 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.490183 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.491719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:46.502150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd23128-2f3c-465a-a84d-a253da801040 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.561981 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.562482 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31c261-c9b8-4777-9634-c345d2525c63 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.563456 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.563919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.564196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.564912 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.566829 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.569075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.569952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.570237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.570512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] PATH_INFO: `/v3/users/16fb5d767a674a2588f87d580b0e317e` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.571747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16fb5d767a674a2588f87d580b0e317e)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.574541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.599956 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1377/2743] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:46 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:46.601060 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.613275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6150f0-117e-4329-8415-c51bbecc1da1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.618897 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.619664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.620020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.620902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.621643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.623468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.632452 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16fb5d767a674a2588f87d580b0e317e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:46.640668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1378/2744] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:46.642937 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.644137 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16fb5d767a674a2588f87d580b0e317e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:46.644910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16fb5d767a674a2588f87d580b0e317e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:46.654511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80191e78-3d01-4716-be76-81d3ba336bca admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.655038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16fb5d767a674a2588f87d580b0e317e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:46.655868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-e741898a-da6a-4e9e-9dcc-fdd3a7cc4ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16fb5d767a674a2588f87d580b0e317e'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:46.659348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.660229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.660514 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.660865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] PATH_INFO: `/v3/projects/2f6dcabb291745e5820de12dfe8b92f9/users/cbaa15f61af6467c974e66562fc6f0b1/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.675086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1367/2745] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:46 2022] DELETE /identity/v3/users/16fb5d767a674a2588f87d580b0e317e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:46.676739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6dcabb291745e5820de12dfe8b92f9, user_id=cbaa15f61af6467c974e66562fc6f0b1, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.679357 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.679542 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.679667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.679918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.698050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1379/2746] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:46 2022] PUT /identity/v3/projects/2f6dcabb291745e5820de12dfe8b92f9/users/cbaa15f61af6467c974e66562fc6f0b1/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:46.698232 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:46.699498 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.700040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:46.701191 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:46.762468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1368/2747] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:46.771726 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59caeab6-dd77-474e-b3f4-33928e39327e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.772344 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.773691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:46.776372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.777173 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.777457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.777732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.778481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.780072 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.790322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1380/2748] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:46.794827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:46.804239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21298a84-878d-4fc7-9ccf-25609db3f29e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.809675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.810535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.810812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.811082 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] PATH_INFO: `/v3/projects/2f6dcabb291745e5820de12dfe8b92f9/users/cbaa15f61af6467c974e66562fc6f0b1/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.813077 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6dcabb291745e5820de12dfe8b92f9, user_id=cbaa15f61af6467c974e66562fc6f0b1, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.815770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.826186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1381/2749] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:46 2022] PUT /identity/v3/projects/2f6dcabb291745e5820de12dfe8b92f9/users/cbaa15f61af6467c974e66562fc6f0b1/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:46.830905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d3ac-52b1-493c-85b6-676c58c75a54 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:46.879302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:46.879936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:46.880219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:46.880495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:46.942488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:46.944116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:46.955320 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1369/2750] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:47.405401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:47.408102 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:47.432893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659eb7d-b655-4886-afca-1abb4476b5a7 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:47.438432 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:47.439357 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:47.439644 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:47.440049 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] PATH_INFO: `/v3/projects/c1d19e0cb15b4432bcb73d1bdb59596c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:47.441415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d19e0cb15b4432bcb73d1bdb59596c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:47.443431 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:47.492831 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d19e0cb15b4432bcb73d1bdb59596c', 'request_id': 'req-e3e1f083-777d-4fb9-8f98-568e5bc5e321'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.499131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d19e0cb15b4432bcb73d1bdb59596c', 'request_id': 'req-e3e1f083-777d-4fb9-8f98-568e5bc5e321'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.508226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d19e0cb15b4432bcb73d1bdb59596c', 'request_id': 'req-e3e1f083-777d-4fb9-8f98-568e5bc5e321'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.528218 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] The token cache is being invalidate because project c1d19e0cb15b4432bcb73d1bdb59596c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:47.528893 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.529658 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.532181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1382/2751] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:47 2022] DELETE /identity/v3/projects/c1d19e0cb15b4432bcb73d1bdb59596c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:47.536317 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:47.548682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f61b0d-55a7-42ab-9914-9553cde4e9ed service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:47.617756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:47.618682 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:47.618962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:47.619238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] PATH_INFO: `/v3/users/ea288ce6ba9e47769a9eab660905b53c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:47.621165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea288ce6ba9e47769a9eab660905b53c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:47.623286 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:47.677764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea288ce6ba9e47769a9eab660905b53c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.686405 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea288ce6ba9e47769a9eab660905b53c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.687170 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea288ce6ba9e47769a9eab660905b53c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.696157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea288ce6ba9e47769a9eab660905b53c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.696797 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-fd6794d8-e131-46bc-b451-f451aefb18c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea288ce6ba9e47769a9eab660905b53c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:47.714863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1370/2752] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:47 2022] DELETE /identity/v3/users/ea288ce6ba9e47769a9eab660905b53c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:48.030819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.032358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:48.055730 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1f083-777d-4fb9-8f98-568e5bc5e321 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.127535 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.128377 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.128870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.129094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] PATH_INFO: `/v3/projects/9c78a01c3f894cf8b56a7822ed8e16bf` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.130287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c78a01c3f894cf8b56a7822ed8e16bf)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.132186 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.180762 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c78a01c3f894cf8b56a7822ed8e16bf', 'request_id': 'req-58e8b9a0-335e-44ed-b553-a76115d87ba8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:48.186437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c78a01c3f894cf8b56a7822ed8e16bf', 'request_id': 'req-58e8b9a0-335e-44ed-b553-a76115d87ba8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:48.195371 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c78a01c3f894cf8b56a7822ed8e16bf', 'request_id': 'req-58e8b9a0-335e-44ed-b553-a76115d87ba8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:48.209075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] The token cache is being invalidate because project 9c78a01c3f894cf8b56a7822ed8e16bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:48.209394 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:48.210832 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-58e8b9a0-335e-44ed-b553-a76115d87ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:48.212625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1383/2753] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:48 2022] DELETE /identity/v3/projects/9c78a01c3f894cf8b56a7822ed8e16bf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:48.218543 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.219251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.219619 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.240859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:48.243247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:48.313458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1371/2754] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:48.394813 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.395105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.395390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.417750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:48.419948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:48.428671 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1384/2755] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:48.432928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.443089 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d501c9-9786-4b43-8539-dc1bd2ca6f82 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.448308 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.449673 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.450202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.450202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.450946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:32:48.451247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.452864 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.464749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1372/2756] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:32:48.469516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.478939 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77053e45-66c6-4367-a8b2-e60e1dd1078e None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.484349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.485153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.485667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.486046 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.486992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.489045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.520045 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1385/2757] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:48 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:48.524882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.537519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46d656-88b8-4578-8e1a-51d9ade4a96f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.598889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.600358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.600849 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.601283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.602114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.605075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.633492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1373/2758] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:48 2022] POST /identity/v3/users => generated 280 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:48.634751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.645933 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c4c7-e602-48e4-ad7e-6b25d51f3e09 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.650680 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.651415 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.651804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.652143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.652959 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.654609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.665986 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1386/2759] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:48.670004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.682179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78059ea9-717f-40c4-9ae6-ecfda7232676 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.689699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b28b6-d4c2-4c29-9c00-87cbc6968318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.689699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b28b6-d4c2-4c29-9c00-87cbc6968318 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.689699 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b28b6-d4c2-4c29-9c00-87cbc6968318 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.690040 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b28b6-d4c2-4c29-9c00-87cbc6968318 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.690576 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b28b6-d4c2-4c29-9c00-87cbc6968318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.692155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b28b6-d4c2-4c29-9c00-87cbc6968318 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.702516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1374/2760] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:32:48.706462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.716964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d64ae-3c14-4001-9acb-47fbe89112f1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:48.723227 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:48.724106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.724693 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.724838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] PATH_INFO: `/v3/projects/f8b49188869e47f9b2eb51d635e24bf2/users/cdcc4fbfd1cf424383dd73c985073fc5/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.743982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b49188869e47f9b2eb51d635e24bf2, user_id=cdcc4fbfd1cf424383dd73c985073fc5, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:48.749260 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:48.766530 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1387/2761] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:48 2022] PUT /identity/v3/projects/f8b49188869e47f9b2eb51d635e24bf2/users/cdcc4fbfd1cf424383dd73c985073fc5/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:48.778869 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:48.779853 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:48.780229 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:48.812361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] MFA Rules not processed for user `cdcc4fbfd1cf424383dd73c985073fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:48.813645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:48.887637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1375/2762] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:48 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:48.926712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:48.929101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:48.996106 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05524b94-3d2a-4457-932e-57ff9a958287 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:49.000202 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4177e-cec5-446a-89b1-277f9fe28503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:49.000965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4177e-cec5-446a-89b1-277f9fe28503 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:49.001312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4177e-cec5-446a-89b1-277f9fe28503 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:49.001657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4177e-cec5-446a-89b1-277f9fe28503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:49.013992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4177e-cec5-446a-89b1-277f9fe28503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:49.016206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4177e-cec5-446a-89b1-277f9fe28503 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:49.027292 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1388/2763] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:48 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:49.043694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:49.142407 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bd71-0adf-4dae-a3b0-1156622c21c7 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:49.150527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:49.152180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:49.152542 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:49.152975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] PATH_INFO: `/v3/users/cdcc4fbfd1cf424383dd73c985073fc5` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:49.154529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdcc4fbfd1cf424383dd73c985073fc5)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:49.156555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:49.244563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdcc4fbfd1cf424383dd73c985073fc5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:49.259535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdcc4fbfd1cf424383dd73c985073fc5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:49.261766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdcc4fbfd1cf424383dd73c985073fc5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:49.275745 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdcc4fbfd1cf424383dd73c985073fc5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:49.277347 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdcc4fbfd1cf424383dd73c985073fc5'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:49.306992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1376/2764] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:32:49 2022] DELETE /identity/v3/users/cdcc4fbfd1cf424383dd73c985073fc5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:49.312170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:49.312459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:49.312826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:49.365764 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:49.367177 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:49.510934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1389/2765] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:49.531150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:49.533897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:49.671439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1e44e-22d7-4afa-be98-bc0c5ac37bc5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:49.677341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dc416-a7a0-448c-8a56-8d274d223f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:49.678098 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc416-a7a0-448c-8a56-8d274d223f7e service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:49.678460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc416-a7a0-448c-8a56-8d274d223f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:49.680146 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc416-a7a0-448c-8a56-8d274d223f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:49.696360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc416-a7a0-448c-8a56-8d274d223f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:49.701086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc416-a7a0-448c-8a56-8d274d223f7e service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:49.745862 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1377/2766] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:50.077761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.079521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:50.108526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46db104-2a92-49f9-ad21-740a0affd8f1 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.123076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.124025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.124306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.125752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] PATH_INFO: `/v3/projects/f8b49188869e47f9b2eb51d635e24bf2` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.134009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b49188869e47f9b2eb51d635e24bf2)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.139136 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:50.260545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b49188869e47f9b2eb51d635e24bf2', 'request_id': 'req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:50.278242 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b49188869e47f9b2eb51d635e24bf2', 'request_id': 'req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:50.288786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b49188869e47f9b2eb51d635e24bf2', 'request_id': 'req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:50.303412 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] The token cache is being invalidate because project f8b49188869e47f9b2eb51d635e24bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:32:50.303713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:50.304471 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-06cf9cf5-01b6-4fee-8296-67ecdef78bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:32:50.306418 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1390/2767] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:32:50 2022] DELETE /identity/v3/projects/f8b49188869e47f9b2eb51d635e24bf2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:32:50.311798 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19102b02-2502-4383-8014-57792b904042 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.312174 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19102b02-2502-4383-8014-57792b904042 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.313400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19102b02-2502-4383-8014-57792b904042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.340172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-19102b02-2502-4383-8014-57792b904042 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:50.342381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-19102b02-2502-4383-8014-57792b904042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:50.470090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1378/2768] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:50.478620 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.478945 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.479179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.508205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:50.509851 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:50.526079 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1391/2769] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:50.531130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19102b02-2502-4383-8014-57792b904042 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.544202 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19102b02-2502-4383-8014-57792b904042 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.559861 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.562144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.562273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.562505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.563372 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:32:50.563755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.566371 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:50.591149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1379/2770] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:32:50.595618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.605281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89110e8-91d5-4d99-900b-d60e37a7c169 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.610093 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.610756 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.611080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.611359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.612056 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.614519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:50.654976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1392/2771] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:32:50 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:32:50.656155 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.668135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dfecd-17a9-4cb5-8679-91411ff7e54c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.740488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.741287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.741507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.742001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.742860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.745201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:50.797855 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1380/2772] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:32:50 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:32:50.801794 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.814964 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b387-2e10-4161-bce5-411ab9790ad0 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.821574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.822689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.823175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.824200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.825323 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.827992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:50.846988 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1393/2773] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:32:50.850637 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.863208 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70aedc-9d3d-4a23-a84d-b7842c41e03a admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.881472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95365f-ac6b-4906-b0d6-7955dfcf36a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.882547 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95365f-ac6b-4906-b0d6-7955dfcf36a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.883063 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95365f-ac6b-4906-b0d6-7955dfcf36a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.883587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95365f-ac6b-4906-b0d6-7955dfcf36a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.884568 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95365f-ac6b-4906-b0d6-7955dfcf36a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.887468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95365f-ac6b-4906-b0d6-7955dfcf36a9 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:50.901523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1381/2774] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:32:50.905968 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:50.927992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90b8f0-b65f-4dc5-afb8-ebcbfa2239be admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:50.953388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:50.958751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:50.959154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:50.959684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] PATH_INFO: `/v3/projects/de285a2e0d3c4ed6b78d7c360cc201fa/users/8232dabfbc794f5686cc24567f41af46/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:50.979731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de285a2e0d3c4ed6b78d7c360cc201fa, user_id=8232dabfbc794f5686cc24567f41af46, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:50.983423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:51.014197 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1394/2775] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:32:50 2022] PUT /identity/v3/projects/de285a2e0d3c4ed6b78d7c360cc201fa/users/8232dabfbc794f5686cc24567f41af46/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:32:51.019764 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37c6fd-c871-4370-8939-3bf55a1263aa None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:51.020058 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37c6fd-c871-4370-8939-3bf55a1263aa None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:51.025118 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37c6fd-c871-4370-8939-3bf55a1263aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:51.067321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-1c37c6fd-c871-4370-8939-3bf55a1263aa None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:51.068812 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-1c37c6fd-c871-4370-8939-3bf55a1263aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:51.201023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1382/2776] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:51.211135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:51.213468 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:51.351888 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f69dd5-c2ea-4226-8549-125c66fc1608 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:51.367587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d7edf-fe30-4704-9967-1bd355101fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:51.368605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d7edf-fe30-4704-9967-1bd355101fcc service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:51.369255 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d7edf-fe30-4704-9967-1bd355101fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:51.369689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d7edf-fe30-4704-9967-1bd355101fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:51.387233 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d7edf-fe30-4704-9967-1bd355101fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:51.389206 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d7edf-fe30-4704-9967-1bd355101fcc service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:51.423346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1395/2777] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:52.617821 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:52.618496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:52.618639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:52.653647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] MFA Rules not processed for user `cbaa15f61af6467c974e66562fc6f0b1`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:52.654873 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:52.674634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:52.675520 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:52.675972 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:52.712286 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] MFA Rules not processed for user `8232dabfbc794f5686cc24567f41af46`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:52.713702 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:52.770820 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1383/2778] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:52.791130 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:52.793032 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:52.812880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1396/2779] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:52.824523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:52.826295 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:52.891157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a529790-75d1-4dd1-9b50-4b1196916dda None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:52.898710 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:52.899685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:52.900172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:52.900704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:52.902151 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24675f-e9f8-4a8a-8edc-b2b6981ea827 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:52.903677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:52.906189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed4e9d-a644-440b-b2f5-208d6023714e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:52.906850 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed4e9d-a644-440b-b2f5-208d6023714e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:52.907128 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed4e9d-a644-440b-b2f5-208d6023714e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:52.907396 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed4e9d-a644-440b-b2f5-208d6023714e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:52.918209 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed4e9d-a644-440b-b2f5-208d6023714e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:52.919665 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed4e9d-a644-440b-b2f5-208d6023714e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:52.929327 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1397/2780] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:52.983763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:52.985434 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9214d1-efa3-48c0-8853-ce66004ace25 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:52.994683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1384/2781] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:53.985899 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:53.986410 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:53.986518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:54.004572 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] MFA Rules not processed for user `17e016e1e171494c9932df04761fb738`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:54.005805 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:54.012536 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1398/2782] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:55.374383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:55.374826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:55.374826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:55.402340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] MFA Rules not processed for user `8232dabfbc794f5686cc24567f41af46`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:32:55.403554 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:55.421743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1385/2783] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:32:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:32:55.699272 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:55.712977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b3006-94e3-49cc-b8aa-3bb50e9d81ec None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:55.717521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:55.718201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:55.718480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:55.718799 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:55.720776 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:55.731837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:55.733399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:55.742211 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1399/2784] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:32:56.131686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:56.133259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:56.136683 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:56.138328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:56.149552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db904e-b3cd-42b0-b794-df8fa268d29d service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:56.153830 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:56.154527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:56.154812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:56.155086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:56.157148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:56.177919 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:56.179407 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:56.191678 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1400/2785] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:56.218528 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8389-070e-45d7-8a24-f3f38156a0ff None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:56.224043 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:56.224822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:56.225585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:56.225585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:56.227143 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:56.239598 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:56.241381 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:56.253413 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1386/2786] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:32:58.624039 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:32:58.626361 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:58.644930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44d687-16ef-4473-aa92-eea9462c9e41 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:32:58.651068 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:32:58.652451 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:32:58.652819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:32:58.653094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:32:58.654869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:32:58.668426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:32:58.670026 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ad628-9f48-4287-a3fa-b711d7afb6d2 service nova] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:32:58.681504 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1401/2787] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:32:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:33:04.581107 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:04.593575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b3b1f-9f79-4506-846e-13c23bc5b8e7 service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:04.599069 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:04.599735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:04.600017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:04.600296 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:04.602091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:04.614196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:04.615762 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:04.626936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1387/2788] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:33:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:33:11.900641 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:11.901152 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:11.901270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:11.918317 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] MFA Rules not processed for user `8232dabfbc794f5686cc24567f41af46`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:33:11.919481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:11.926160 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1402/2789] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:33:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:33:27.345354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:27.356579 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872970e-6123-4108-81bd-870cbb8b2146 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:27.366684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:27.367601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:27.368006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:27.368395 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:27.372353 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:27.386702 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:27.388912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489d112-a398-449a-a45a-c0667f09657f service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:27.400363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1388/2790] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:33:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:33:34.889212 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:34.898552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac4661-4ec4-4eb2-9fc5-1e2f6f9642b5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:34.902381 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:34.903025 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:34.903306 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:34.903582 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:34.913343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:34.915067 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:34.925948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1403/2791] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:33:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:33:35.229219 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:35.229661 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:35.229754 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:35.288072 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:33:35.289316 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:35.355334 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1389/2792] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 19:33:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:33:35.407105 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:35.408475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:35.499533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d9ca-da86-4d6e-9262-ef5f43451126 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:35.504656 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a735f-1597-4eb2-a0ee-877ae2c03aed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:35.505310 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a735f-1597-4eb2-a0ee-877ae2c03aed service swift] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:35.505592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a735f-1597-4eb2-a0ee-877ae2c03aed service swift] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:35.505896 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a735f-1597-4eb2-a0ee-877ae2c03aed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:35.514881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a735f-1597-4eb2-a0ee-877ae2c03aed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:35.516457 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a735f-1597-4eb2-a0ee-877ae2c03aed service swift] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:35.524263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1404/2793] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 16 19:33:37.134727 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:37.148213 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff87e5-4e15-48ee-a088-6ff532805798 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:37.156801 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:37.158203 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:37.158846 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:37.159573 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:37.163982 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:37.197882 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:37.200083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:37.211374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1390/2794] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:33:51.703840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:51.704299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:51.704389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:51.722331 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:33:51.723511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:33:51.730746 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1405/2795] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 16 19:33:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:33:51.743103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:51.756418 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50011327-da64-4915-8e80-79d08b1cce3a service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:51.761398 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:51.762038 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:51.762350 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:51.762666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:51.770730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:51.772301 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:51.780608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1391/2796] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:33:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 16 19:33:53.655474 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:53.666831 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3ee0b-189a-43b2-8ce0-2e211288c0b6 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:53.671263 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:53.672126 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:53.672401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:53.672697 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] PATH_INFO: `/v3/users/8232dabfbc794f5686cc24567f41af46` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:53.673872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8232dabfbc794f5686cc24567f41af46)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:53.675748 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:53.721892 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8232dabfbc794f5686cc24567f41af46'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:53.722445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8232dabfbc794f5686cc24567f41af46'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:53.729034 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8232dabfbc794f5686cc24567f41af46'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:53.736058 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8232dabfbc794f5686cc24567f41af46'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:53.736727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8232dabfbc794f5686cc24567f41af46'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:53.753722 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1406/2797] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:33:53 2022] DELETE /identity/v3/users/8232dabfbc794f5686cc24567f41af46 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:33:53.928520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:33:53.940410 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcae67a-3ea0-4376-971a-6d946bf24eef service swift] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:33:53.999472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:33:54.000680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:33:54.001009 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:33:54.001300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] PATH_INFO: `/v3/projects/de285a2e0d3c4ed6b78d7c360cc201fa` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:33:54.002572 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de285a2e0d3c4ed6b78d7c360cc201fa)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:33:54.004448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:33:54.050090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de285a2e0d3c4ed6b78d7c360cc201fa', 'request_id': 'req-2e303921-889d-4c24-8316-e4e1cbc0737f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:54.055147 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de285a2e0d3c4ed6b78d7c360cc201fa', 'request_id': 'req-2e303921-889d-4c24-8316-e4e1cbc0737f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:54.060939 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de285a2e0d3c4ed6b78d7c360cc201fa', 'request_id': 'req-2e303921-889d-4c24-8316-e4e1cbc0737f'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:54.073235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] The token cache is being invalidate because project de285a2e0d3c4ed6b78d7c360cc201fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:33:54.073512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:54.074103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:33:54.075622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1392/2798] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:33:53 2022] DELETE /identity/v3/projects/de285a2e0d3c4ed6b78d7c360cc201fa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:34:31.806977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:34:31.808313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:34:31.867733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eaf55-ae6d-4718-9fe9-db5a9647002a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:34:31.918204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:34:31.918842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:34:31.919119 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:34:31.919388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:34:31.921200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:34:32.003470 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:34:32.004985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:34:32.022258 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1407/2799] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:34:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:06.375639 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:06.377103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:06.431427 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e303921-889d-4c24-8316-e4e1cbc0737f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:06.435039 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:06.435665 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:06.435941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:06.436211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:06.437936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:06.503087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:06.504492 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:06.514935 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1393/2800] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:18.321335 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:18.322871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:18.335035 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84c4df-b65f-424a-ae16-1f523cc36505 service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:18.338526 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:18.339148 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:18.339421 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:18.339689 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:18.341395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:18.351084 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:18.352486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:18.360120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1408/2801] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:36:18.459294 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:18.468378 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d8946-fd3a-4d4b-b993-03ef37a1ae50 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:18.477266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:18.477964 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:18.478247 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:18.478521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:18.480791 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:18.492374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:18.493929 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:18.503382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1394/2802] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:18.604567 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:18.615291 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241421a3-7300-4d6d-bd8b-cff19206a8d3 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:18.620349 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:18.621091 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:18.621434 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:18.621703 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:18.623825 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:18.638030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:18.639436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:18.650063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1409/2803] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:36:32.591649 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:32.593103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:32.716522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990e400-414a-4657-aa66-fbdae6712d99 service neutron] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:32.721603 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:32.722237 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:32.722522 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:32.722792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:32.725148 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:32.736121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:32.737718 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:32.747705 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1395/2804] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:39.890172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:39.898570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743af91-6249-4e9e-9a1e-fb0ec94c180f service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:39.902877 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:39.903587 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:39.903873 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:39.904155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:39.906000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:39.967674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:39.969156 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:39.978812 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1410/2805] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:43.013023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.015142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:43.030555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb4bd1-e687-4ff7-aa79-503fc9c3f69a service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.035092 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.035958 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.036236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.036507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] PATH_INFO: `/v3/users/e5c547a57e5e44fb8a1c89558b480991` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.037766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c547a57e5e44fb8a1c89558b480991)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.039674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.084903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c547a57e5e44fb8a1c89558b480991'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.091472 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c547a57e5e44fb8a1c89558b480991'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.092116 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c547a57e5e44fb8a1c89558b480991'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.098537 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c547a57e5e44fb8a1c89558b480991'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.099254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-14d1b18b-6993-4171-bf4a-7289653f9980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c547a57e5e44fb8a1c89558b480991'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.115627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1396/2806] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:36:43 2022] DELETE /identity/v3/users/e5c547a57e5e44fb8a1c89558b480991 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:36:43.285383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.301251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f1298-82cd-4c94-b48e-351b9ad1135b service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.361681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.362557 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.362838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.363137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] PATH_INFO: `/v3/projects/5c80ec96b2f34c42a02d886b574e37c0` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.364307 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c80ec96b2f34c42a02d886b574e37c0)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.366401 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.410858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c80ec96b2f34c42a02d886b574e37c0', 'request_id': 'req-1b39ab36-a1b1-4ab6-b815-43e2c509d123'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.416540 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c80ec96b2f34c42a02d886b574e37c0', 'request_id': 'req-1b39ab36-a1b1-4ab6-b815-43e2c509d123'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.423907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c80ec96b2f34c42a02d886b574e37c0', 'request_id': 'req-1b39ab36-a1b1-4ab6-b815-43e2c509d123'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.434954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] The token cache is being invalidate because project 5c80ec96b2f34c42a02d886b574e37c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:36:43.435237 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.435852 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-1b39ab36-a1b1-4ab6-b815-43e2c509d123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:43.437375 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1411/2807] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:36:43 2022] DELETE /identity/v3/projects/5c80ec96b2f34c42a02d886b574e37c0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:43.442994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.443420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.443819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.463545 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:43.464740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:43.521983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1397/2808] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:43.530023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.530245 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.530523 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.548903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:43.550083 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:43.558446 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1412/2809] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:43.562385 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.571990 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743f2c-9aa6-48b6-901b-f479ce429842 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.576517 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.577183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.577462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.577734 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] PATH_INFO: `/v3/domains` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.578442 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91543) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:36:43.578744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.580342 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.591266 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1398/2810] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:36:43.595253 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.604625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cd343-353f-4d5b-914f-aa9bb8961631 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.609146 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.609779 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.610055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.610390 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.611073 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.612942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.642131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1413/2811] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:36:43 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:36:43.643341 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.653685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7ea4a-a498-4f46-8fd3-365548d5f07b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.710597 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.711256 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.711534 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.712323 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.713032 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.714918 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.735969 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1399/2812] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:36:43 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:36:43.740200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.752248 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff3238-f257-42eb-ad72-1605b522201a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.756835 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.757475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.757752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.758020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.758727 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.760289 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.770544 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1414/2813] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:36:43.774947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.784826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e69377-4b3d-450d-8480-4bdaa0cbece5 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.789292 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904c28-b537-4400-a12f-0b39c5ae7a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.789977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904c28-b537-4400-a12f-0b39c5ae7a19 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.790792 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904c28-b537-4400-a12f-0b39c5ae7a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.791064 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904c28-b537-4400-a12f-0b39c5ae7a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.791779 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73904c28-b537-4400-a12f-0b39c5ae7a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.793644 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73904c28-b537-4400-a12f-0b39c5ae7a19 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.803622 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1400/2814] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:36:43.807309 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:43.817043 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2d3aa-3aba-417e-b8d3-945a5cf5b4e4 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:43.821690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:43.822581 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.822861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.823141 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] PATH_INFO: `/v3/projects/8702d71f47c2472a81d0ffafd70ac63a/users/afe170279f654bddb44e6e1afaeb75d3/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.838235 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8702d71f47c2472a81d0ffafd70ac63a, user_id=afe170279f654bddb44e6e1afaeb75d3, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:43.840870 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:43.853958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1415/2815] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:43 2022] PUT /identity/v3/projects/8702d71f47c2472a81d0ffafd70ac63a/users/afe170279f654bddb44e6e1afaeb75d3/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:36:43.865076 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:43.865376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:43.865650 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:43.892592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] MFA Rules not processed for user `afe170279f654bddb44e6e1afaeb75d3`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:43.893813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:43.957476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1401/2816] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:43.961951 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.020455 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc35e6a-0c4e-4598-837c-3d1c077b76e7 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.025135 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.025777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.026057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.026328 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.027008 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.028911 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.053032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1416/2817] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:36:43 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:36:44.057510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.067881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57e77f-1c5f-4696-9880-3b2276fba38f None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.125868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.126529 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.126806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.127080 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] PATH_INFO: `/v3/users` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.127919 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.129844 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.149406 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1402/2818] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:36:44 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:36:44.153287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.163249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23d7c-4f85-40f1-a57e-151010102cc7 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.168516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.169194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.169475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.169750 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.170464 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.172070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.182732 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1417/2819] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:44.186582 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.196061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b0822-2d2d-4b66-8edf-8083450aaa85 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.200704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.201561 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.201833 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.202103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] PATH_INFO: `/v3/projects/9c0c0e327a034732ba44404a68969a5d/users/08f71ef90fee44afa8f6b93c15682415/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.215790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0c0e327a034732ba44404a68969a5d, user_id=08f71ef90fee44afa8f6b93c15682415, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.218424 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.231585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1403/2820] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:44 2022] PUT /identity/v3/projects/9c0c0e327a034732ba44404a68969a5d/users/08f71ef90fee44afa8f6b93c15682415/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:44.235990 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.247028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528922e-f50c-443c-841a-568e3a9f254b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.300213 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c354ed-0b50-4cfa-a120-9a1933ee43bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.300916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c354ed-0b50-4cfa-a120-9a1933ee43bf admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.301196 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c354ed-0b50-4cfa-a120-9a1933ee43bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.301467 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c354ed-0b50-4cfa-a120-9a1933ee43bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.302176 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c354ed-0b50-4cfa-a120-9a1933ee43bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.303886 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c354ed-0b50-4cfa-a120-9a1933ee43bf admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.313780 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1418/2821] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:44.317867 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.327723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163da9a3-b65f-4d05-9017-38f9d61e3779 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.332176 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.333068 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.333345 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.333617 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] PATH_INFO: `/v3/projects/9c0c0e327a034732ba44404a68969a5d/users/08f71ef90fee44afa8f6b93c15682415/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.335541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0c0e327a034732ba44404a68969a5d, user_id=08f71ef90fee44afa8f6b93c15682415, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.338163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.349488 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1404/2822] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:44 2022] PUT /identity/v3/projects/9c0c0e327a034732ba44404a68969a5d/users/08f71ef90fee44afa8f6b93c15682415/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:44.359868 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.360153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.360426 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.386054 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] MFA Rules not processed for user `08f71ef90fee44afa8f6b93c15682415`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:44.387239 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:44.452199 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1419/2823] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:44.463233 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:44.464973 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:44.519435 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845f208-5b9d-4d5c-b515-10933f34dc04 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:44.523947 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59953e-c954-49c7-b0f0-cf73a768aec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:44.525142 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59953e-c954-49c7-b0f0-cf73a768aec5 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:44.525459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59953e-c954-49c7-b0f0-cf73a768aec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:44.525733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59953e-c954-49c7-b0f0-cf73a768aec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:44.534926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59953e-c954-49c7-b0f0-cf73a768aec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:44.536363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59953e-c954-49c7-b0f0-cf73a768aec5 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:44.546737 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1405/2824] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:45.801636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:45.861443 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10208c39-d331-4de6-bbf0-f9fc3da98d3d None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:45.865761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:45.866610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:45.866883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:45.867149 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] PATH_INFO: `/v3/users/afe170279f654bddb44e6e1afaeb75d3` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:45.868548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe170279f654bddb44e6e1afaeb75d3)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:45.870547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:45.916822 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe170279f654bddb44e6e1afaeb75d3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:45.917719 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe170279f654bddb44e6e1afaeb75d3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:45.924383 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe170279f654bddb44e6e1afaeb75d3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:45.932901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe170279f654bddb44e6e1afaeb75d3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:45.933594 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe170279f654bddb44e6e1afaeb75d3'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:45.951577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1420/2825] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:36:45 2022] DELETE /identity/v3/users/afe170279f654bddb44e6e1afaeb75d3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:45.956183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:45.956747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:45.956877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:45.974191 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:45.975382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:46.031431 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1406/2826] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:46.042715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:46.045140 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:46.105889 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453d1b2-f783-47ec-a885-d37b3e99b5fa admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:46.109480 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:46.110109 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:46.110389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:46.110660 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:46.119921 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:46.121302 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:46.130643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1421/2827] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:46.393419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:46.394822 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:46.406944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0242a0-132a-4218-88ab-a14e35f80d44 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:46.411559 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:46.412420 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:46.412729 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:46.413001 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] PATH_INFO: `/v3/projects/8702d71f47c2472a81d0ffafd70ac63a` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:46.414175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8702d71f47c2472a81d0ffafd70ac63a)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:46.416060 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:46.462740 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8702d71f47c2472a81d0ffafd70ac63a', 'request_id': 'req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.467984 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8702d71f47c2472a81d0ffafd70ac63a', 'request_id': 'req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.473936 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8702d71f47c2472a81d0ffafd70ac63a', 'request_id': 'req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.486196 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] The token cache is being invalidate because project 8702d71f47c2472a81d0ffafd70ac63a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:36:46.486701 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.487430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.489336 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1407/2828] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:36:46 2022] DELETE /identity/v3/projects/8702d71f47c2472a81d0ffafd70ac63a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:46.493638 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:46.504657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df221c4-bc49-405d-9276-4e7762dd9e25 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:46.558399 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:46.559281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:46.559543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:46.559814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] PATH_INFO: `/v3/users/08f71ef90fee44afa8f6b93c15682415` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:46.561071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f71ef90fee44afa8f6b93c15682415)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:46.563000 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:46.613580 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f71ef90fee44afa8f6b93c15682415'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.614236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f71ef90fee44afa8f6b93c15682415'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.622558 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f71ef90fee44afa8f6b93c15682415'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.631597 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f71ef90fee44afa8f6b93c15682415'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.632409 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-cc4e6c6c-b03e-44d4-bbed-9cc97f42aac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f71ef90fee44afa8f6b93c15682415'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:46.655503 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1422/2829] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:36:46 2022] DELETE /identity/v3/users/08f71ef90fee44afa8f6b93c15682415 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:36:46.935227 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:46.937396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:46.953977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f204-cd68-4c3b-a2bf-16c81d012e3c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.022363 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.023221 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.023496 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.023767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] PATH_INFO: `/v3/projects/9c0c0e327a034732ba44404a68969a5d` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.024987 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0c0e327a034732ba44404a68969a5d)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.026860 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.072312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0c0e327a034732ba44404a68969a5d', 'request_id': 'req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:47.077541 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0c0e327a034732ba44404a68969a5d', 'request_id': 'req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:47.083382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0c0e327a034732ba44404a68969a5d', 'request_id': 'req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:47.095127 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] The token cache is being invalidate because project 9c0c0e327a034732ba44404a68969a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:36:47.095430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:47.096032 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-1fdb1f63-daf3-4d7b-a398-afe3fd5919b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:47.097585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1408/2830] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:36:46 2022] DELETE /identity/v3/projects/9c0c0e327a034732ba44404a68969a5d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:47.102954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.103484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.103484 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.121121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:47.122313 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:47.178992 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1423/2831] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:47.187282 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.187558 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.187830 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.205691 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:47.206866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:47.214731 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1409/2832] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:36:47.218918 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.228936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d049bb-73ed-4ff7-8628-064a6e96069a None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.233688 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.234440 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.234721 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.234995 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.235749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:36:47.236063 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.238181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.252365 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1424/2833] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:36:47.257029 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.266103 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd2b5-bfb6-473a-9739-f28605e5d699 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.270904 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.271544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.271819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.272088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.272800 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.274680 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.300126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1410/2834] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/projects => generated 407 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:36:47.304380 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.314714 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a22ef-91ab-483e-8f63-76c2aeec7a87 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.375713 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.376712 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.377134 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.377543 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.378589 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.381420 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.402944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1425/2835] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:36:47.407035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.417826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417813-1f0b-421e-8329-d3e65a25d887 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.422355 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.423002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.423283 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.423555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.424273 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.425893 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.438629 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1411/2836] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:47.442353 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.452527 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3113a-c837-478f-8886-5ecae126e925 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.457373 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7988da-45bf-478b-8be1-cce330c25d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.458020 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7988da-45bf-478b-8be1-cce330c25d59 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.458296 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7988da-45bf-478b-8be1-cce330c25d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.458564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7988da-45bf-478b-8be1-cce330c25d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.459275 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7988da-45bf-478b-8be1-cce330c25d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.460902 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7988da-45bf-478b-8be1-cce330c25d59 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.470976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1426/2837] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:47.475530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.486302 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff437e8-b08b-4bf0-97bc-addd5b0c3a5d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.490962 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.492172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.492448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.492763 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] PATH_INFO: `/v3/projects/38af7cfb25f14f26893b4b3316c10256/users/86a2d08cb9a24c1e94d60ca34700ca3c/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.507520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38af7cfb25f14f26893b4b3316c10256, user_id=86a2d08cb9a24c1e94d60ca34700ca3c, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.510186 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.522876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1412/2838] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:47 2022] PUT /identity/v3/projects/38af7cfb25f14f26893b4b3316c10256/users/86a2d08cb9a24c1e94d60ca34700ca3c/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:47.534507 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.534823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.535131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.561668 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] MFA Rules not processed for user `86a2d08cb9a24c1e94d60ca34700ca3c`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:47.562865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:47.624355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1427/2839] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:36:47.628868 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.687422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c550267-31c5-424a-9b8d-49961b4e5e1c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.692091 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.692711 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.692991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.693260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.693938 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.695817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.721505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1413/2840] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/projects => generated 409 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:36:47.725518 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.736733 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24f1ff-8d00-44b6-9976-ec35d03e7e18 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.797465 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.798139 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.798417 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.798686 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.799461 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.801513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.826834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1428/2841] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:36:47 2022] POST /identity/v3/users => generated 311 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:36:47.830744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.841992 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea2df-cb5a-46d0-91fc-1ee73a2793a7 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.846521 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.847245 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.847524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.847799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.848877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.850460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.862749 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1414/2842] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:47.866859 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.876795 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687302-621b-489f-92e4-0cf765df45e1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.881354 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.882214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.882494 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.882763 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] PATH_INFO: `/v3/projects/209cac51e16542c38ad8ed1720f6759c/users/94c18a37f0f54f9a8d2dc5fc161c1f58/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:47.902080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209cac51e16542c38ad8ed1720f6759c, user_id=94c18a37f0f54f9a8d2dc5fc161c1f58, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:47.907142 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:47.927131 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1429/2843] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:47 2022] PUT /identity/v3/projects/209cac51e16542c38ad8ed1720f6759c/users/94c18a37f0f54f9a8d2dc5fc161c1f58/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:47.932396 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:47.942159 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b232-0e0c-4d18-8adf-35ab0102a70e admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:47.998197 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38fe81-4025-485f-8fc4-aa36f29c8b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:47.998852 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38fe81-4025-485f-8fc4-aa36f29c8b16 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:47.999183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38fe81-4025-485f-8fc4-aa36f29c8b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:47.999462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38fe81-4025-485f-8fc4-aa36f29c8b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:48.000223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38fe81-4025-485f-8fc4-aa36f29c8b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:48.002225 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38fe81-4025-485f-8fc4-aa36f29c8b16 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:48.012963 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1415/2844] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:48.017492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:48.028574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340427b-17f5-4032-9ace-393fc479b525 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:48.033270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:48.034935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] REQUEST_METHOD: `PUT` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:48.034935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:48.034935 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] PATH_INFO: `/v3/projects/209cac51e16542c38ad8ed1720f6759c/users/94c18a37f0f54f9a8d2dc5fc161c1f58/roles/461cbc289f54486bb7c5bb877fa01923` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:48.036749 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] RBAC: Authorizing `identity:create_grant(project_id=209cac51e16542c38ad8ed1720f6759c, user_id=94c18a37f0f54f9a8d2dc5fc161c1f58, role_id=461cbc289f54486bb7c5bb877fa01923)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:48.039633 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:48.053009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1430/2845] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:48 2022] PUT /identity/v3/projects/209cac51e16542c38ad8ed1720f6759c/users/94c18a37f0f54f9a8d2dc5fc161c1f58/roles/461cbc289f54486bb7c5bb877fa01923 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:48.065228 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:48.065781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:48.065781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:48.102295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] MFA Rules not processed for user `94c18a37f0f54f9a8d2dc5fc161c1f58`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:48.103602 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:48.178587 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1416/2846] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:48 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:36:48.189861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:48.191639 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:48.250862 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e33b9-5b40-4845-85cf-29e32a97b52f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:48.254800 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:48.255436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:48.255715 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:48.255987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:48.265627 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:48.267042 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:48.276204 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1431/2847] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:48.426466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:48.436476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5d7dd-1801-4445-b520-1f64fb0078a3 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:48.439944 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3424a-5a41-41a9-8939-fec19a82f234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:48.440567 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3424a-5a41-41a9-8939-fec19a82f234 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:48.440876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3424a-5a41-41a9-8939-fec19a82f234 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:48.441149 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3424a-5a41-41a9-8939-fec19a82f234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:48.495681 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3424a-5a41-41a9-8939-fec19a82f234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:48.497136 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3424a-5a41-41a9-8939-fec19a82f234 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:48.505877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1417/2848] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:57.780454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:57.838575 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6b984-b992-4e01-b120-47643bcd638e service cinder] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:57.843075 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:57.843914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:57.844189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:57.844459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] PATH_INFO: `/v3/users/86a2d08cb9a24c1e94d60ca34700ca3c` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:57.845631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a2d08cb9a24c1e94d60ca34700ca3c)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:57.847545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:57.891752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a2d08cb9a24c1e94d60ca34700ca3c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:57.892315 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a2d08cb9a24c1e94d60ca34700ca3c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:57.898866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a2d08cb9a24c1e94d60ca34700ca3c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:57.905866 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a2d08cb9a24c1e94d60ca34700ca3c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:57.906120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a2d08cb9a24c1e94d60ca34700ca3c'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:57.923259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1432/2849] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:36:57 2022] DELETE /identity/v3/users/86a2d08cb9a24c1e94d60ca34700ca3c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:57.927907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:57.928458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:57.928550 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:57.945563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:57.946723 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:58.003590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1418/2850] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:58.013179 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:58.014568 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:58.071977 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c2e5e-0e44-4fcc-8a9c-02ce6a565a26 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:58.075806 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:58.076441 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:58.076771 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:58.077051 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:58.085752 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:58.087155 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:58.096208 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1433/2851] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:58.359059 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:58.360464 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:58.372959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0b648-0e56-42ae-92ba-3b2ca23aef72 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:58.377520 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:58.378368 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:58.378642 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:58.378910 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] PATH_INFO: `/v3/projects/38af7cfb25f14f26893b4b3316c10256` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:58.380086 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38af7cfb25f14f26893b4b3316c10256)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:58.382006 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:58.427238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38af7cfb25f14f26893b4b3316c10256', 'request_id': 'req-6247872b-c194-4d4a-adf3-5b0b7935ba62'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.432409 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38af7cfb25f14f26893b4b3316c10256', 'request_id': 'req-6247872b-c194-4d4a-adf3-5b0b7935ba62'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.438451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38af7cfb25f14f26893b4b3316c10256', 'request_id': 'req-6247872b-c194-4d4a-adf3-5b0b7935ba62'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.450270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] The token cache is being invalidate because project 38af7cfb25f14f26893b4b3316c10256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:36:58.450500 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.451129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.452663 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1419/2852] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:36:58 2022] DELETE /identity/v3/projects/38af7cfb25f14f26893b4b3316c10256 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:58.456519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:58.466423 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3feccd-2d3e-4cd6-81ad-f24dec539721 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:58.518502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:58.519344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:58.519618 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:58.519881 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] PATH_INFO: `/v3/users/94c18a37f0f54f9a8d2dc5fc161c1f58` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:58.521059 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c18a37f0f54f9a8d2dc5fc161c1f58)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:58.522928 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:58.565701 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c18a37f0f54f9a8d2dc5fc161c1f58'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.566325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c18a37f0f54f9a8d2dc5fc161c1f58'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.572687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c18a37f0f54f9a8d2dc5fc161c1f58'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.579706 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c18a37f0f54f9a8d2dc5fc161c1f58'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.580322 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-3358ce04-9b44-4c8e-a1b3-fd2e22dad7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c18a37f0f54f9a8d2dc5fc161c1f58'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.597339 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1434/2853] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:36:58 2022] DELETE /identity/v3/users/94c18a37f0f54f9a8d2dc5fc161c1f58 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:58.857090 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:58.858755 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:58.872758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247872b-c194-4d4a-adf3-5b0b7935ba62 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:58.927526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:58.928387 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:58.928966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:58.928966 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] PATH_INFO: `/v3/projects/209cac51e16542c38ad8ed1720f6759c` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:58.930129 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209cac51e16542c38ad8ed1720f6759c)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:58.932022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:58.976902 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209cac51e16542c38ad8ed1720f6759c', 'request_id': 'req-80690b36-0e2d-4a5e-9ffd-1669787976f7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.982817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209cac51e16542c38ad8ed1720f6759c', 'request_id': 'req-80690b36-0e2d-4a5e-9ffd-1669787976f7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:58.988926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209cac51e16542c38ad8ed1720f6759c', 'request_id': 'req-80690b36-0e2d-4a5e-9ffd-1669787976f7'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.000975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] The token cache is being invalidate because project 209cac51e16542c38ad8ed1720f6759c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:36:59.001254 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.001872 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-80690b36-0e2d-4a5e-9ffd-1669787976f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.003394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1420/2854] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:36:58 2022] DELETE /identity/v3/projects/209cac51e16542c38ad8ed1720f6759c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:59.008214 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.008661 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.008819 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.025786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:59.026954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:59.084118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1435/2855] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:59.093369 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.093656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.093933 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.113083 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:59.114252 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:59.121888 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1421/2856] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:59.125824 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.135145 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ad22c-8b8e-43d4-9333-207fba533f96 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.139676 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.140319 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.140600 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.140907 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.141631 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:36:59.141936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.143516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.155563 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1436/2857] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:36:59.159819 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.170018 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751bcc-0ee1-4c04-97e4-2904bfdeab30 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.174817 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.175466 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.175744 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.176017 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.176748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.178735 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.203023 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1422/2858] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:36:59.206844 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.217080 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a730d0-4e45-4d95-9353-6c17b5b6c89f admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.272684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.273282 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.273570 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.273837 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.274510 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.276400 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.296519 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1437/2859] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:36:59 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:36:59.300715 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.310716 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eb9c0-b9b1-496f-bfaf-a5be6752d507 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.315312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.315953 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.316235 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.316507 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.317269 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.318841 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.328778 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1423/2860] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:59.332393 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.342425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f9228-5356-4b74-a732-c651d7558702 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.347010 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94b718-5da8-4f51-bd27-20e5cb2b74eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.347643 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94b718-5da8-4f51-bd27-20e5cb2b74eb admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.347923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94b718-5da8-4f51-bd27-20e5cb2b74eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.348325 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94b718-5da8-4f51-bd27-20e5cb2b74eb admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.348962 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94b718-5da8-4f51-bd27-20e5cb2b74eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.350483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94b718-5da8-4f51-bd27-20e5cb2b74eb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.360653 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1438/2861] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:36:59.364689 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.374719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35960b-8fff-4a8f-b01d-3127e77d417d admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.379231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.380084 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.380358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.380645 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] PATH_INFO: `/v3/projects/3370c29431a84c579a604b933e264682/users/49a925d6d0274b569eee5b2895575fd8/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.394885 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3370c29431a84c579a604b933e264682, user_id=49a925d6d0274b569eee5b2895575fd8, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.397591 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.410459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1424/2862] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:36:59 2022] PUT /identity/v3/projects/3370c29431a84c579a604b933e264682/users/49a925d6d0274b569eee5b2895575fd8/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:36:59.466477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.466760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.467031 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.492130 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] MFA Rules not processed for user `49a925d6d0274b569eee5b2895575fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:59.493336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:59.555299 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1439/2863] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:59.566094 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.567756 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:59.621044 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dba85f-75d7-4e00-8e30-fba1eb10a83b admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.624604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bc7ba-885e-42ed-a511-b21f31e75861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.625259 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bc7ba-885e-42ed-a511-b21f31e75861 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.625535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bc7ba-885e-42ed-a511-b21f31e75861 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.625807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bc7ba-885e-42ed-a511-b21f31e75861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.633813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bc7ba-885e-42ed-a511-b21f31e75861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.635257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bc7ba-885e-42ed-a511-b21f31e75861 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.644249 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1425/2864] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:36:59.663247 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.722251 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834018de-0fc6-4234-aa13-7f39bbcf2eb8 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.726739 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.727585 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.727856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.728121 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] PATH_INFO: `/v3/users/49a925d6d0274b569eee5b2895575fd8` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.729312 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a925d6d0274b569eee5b2895575fd8)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.731200 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.774761 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a925d6d0274b569eee5b2895575fd8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.775414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a925d6d0274b569eee5b2895575fd8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.781987 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a925d6d0274b569eee5b2895575fd8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.789023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a925d6d0274b569eee5b2895575fd8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.789664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a925d6d0274b569eee5b2895575fd8'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:36:59.806549 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1440/2865] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:36:59 2022] DELETE /identity/v3/users/49a925d6d0274b569eee5b2895575fd8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:36:59.811438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.811753 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.812026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.828952 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:36:59.830121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:59.884361 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1426/2866] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:36:59.894595 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:36:59.896002 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:36:59.952845 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56005abd-361d-4fb0-8cba-372c32cbd223 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:36:59.956397 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f424325-193e-4e08-be0d-e9b24773dddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:36:59.957076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f424325-193e-4e08-be0d-e9b24773dddc service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:36:59.957336 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f424325-193e-4e08-be0d-e9b24773dddc service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:36:59.957630 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f424325-193e-4e08-be0d-e9b24773dddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:36:59.966547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f424325-193e-4e08-be0d-e9b24773dddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:36:59.967953 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f424325-193e-4e08-be0d-e9b24773dddc service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:36:59.976957 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1441/2867] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:00.227903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.229346 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:00.242376 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab2f70-e0f2-44f2-b742-0a58b39de13c None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.246991 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.248977 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.249438 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.249739 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] PATH_INFO: `/v3/projects/3370c29431a84c579a604b933e264682` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.250912 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3370c29431a84c579a604b933e264682)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.252825 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.297590 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3370c29431a84c579a604b933e264682', 'request_id': 'req-0b88e1db-efb5-4532-9f24-99f5c23c7964'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:00.302997 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3370c29431a84c579a604b933e264682', 'request_id': 'req-0b88e1db-efb5-4532-9f24-99f5c23c7964'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:00.309161 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3370c29431a84c579a604b933e264682', 'request_id': 'req-0b88e1db-efb5-4532-9f24-99f5c23c7964'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:00.321653 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] The token cache is being invalidate because project 3370c29431a84c579a604b933e264682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:37:00.321876 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:00.322581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-0b88e1db-efb5-4532-9f24-99f5c23c7964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:00.324135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1427/2868] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:37:00 2022] DELETE /identity/v3/projects/3370c29431a84c579a604b933e264682 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:00.329153 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.329667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.329784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.346657 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:00.348070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:00.404963 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1442/2869] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:00.412652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.413019 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.413367 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.431894 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:00.433194 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:00.442373 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1428/2870] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:00.446411 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.455690 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9351-97ea-4cfa-abd7-142296198fb7 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.460150 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.460826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.461107 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.461378 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.462089 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:37:00.462391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.463976 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.475162 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1443/2871] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:37:00.479185 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.488523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbfb5a-ae48-4c8e-9346-fe069ad10bbf None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.493178 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.493806 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.494088 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.494360 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.495031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.496946 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.522157 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1429/2872] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:37:00 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:00.526240 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.536477 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c6051-7ae6-4046-b0ef-437d49d724df admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.592015 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.592605 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.592922 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.593195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.593872 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.595758 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.615032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1444/2873] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:37:00 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:00.619251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.629231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86d650-8745-42d6-a165-8047f29000c6 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.633813 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.634459 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.634748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.635020 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.635742 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.637375 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.647404 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1430/2874] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:00.651090 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.661101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30093ba0-4948-4454-a113-ddab4b25f674 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.665684 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded608d4-3dbd-48d8-b6d3-745b65d7f07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.666320 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded608d4-3dbd-48d8-b6d3-745b65d7f07a admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.666598 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded608d4-3dbd-48d8-b6d3-745b65d7f07a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.666874 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded608d4-3dbd-48d8-b6d3-745b65d7f07a admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.667571 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded608d4-3dbd-48d8-b6d3-745b65d7f07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.670028 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded608d4-3dbd-48d8-b6d3-745b65d7f07a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.679778 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1445/2875] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:00.683767 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.693502 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cfd00-42c9-47ef-97fe-f5bbfe73cfb3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.698021 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.698875 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.699152 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.699422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] PATH_INFO: `/v3/projects/97c79be6f62540e09e602c73c6f4ecb5/users/a5e021938485423d8c310dd2f0128518/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.712994 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c79be6f62540e09e602c73c6f4ecb5, user_id=a5e021938485423d8c310dd2f0128518, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.715577 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.727829 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1431/2876] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:37:00 2022] PUT /identity/v3/projects/97c79be6f62540e09e602c73c6f4ecb5/users/a5e021938485423d8c310dd2f0128518/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:37:00.739485 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.739777 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.740052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.767548 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] MFA Rules not processed for user `a5e021938485423d8c310dd2f0128518`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:00.769364 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:00.832978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1446/2877] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:37:00.845025 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:00.846714 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:00.901253 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d5143-d347-4969-b860-7a0b61f85dca admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:00.905505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2f211-4503-4a04-8e61-6ae596566de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:00.906139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2f211-4503-4a04-8e61-6ae596566de6 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:00.906419 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2f211-4503-4a04-8e61-6ae596566de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:00.906695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2f211-4503-4a04-8e61-6ae596566de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:00.916099 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2f211-4503-4a04-8e61-6ae596566de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:00.917555 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2f211-4503-4a04-8e61-6ae596566de6 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:00.926556 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1432/2878] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:00.945681 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.006436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708ebe-4365-48a3-91c4-8fc847524624 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.010978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.011826 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.012103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.012391 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] PATH_INFO: `/v3/users/a5e021938485423d8c310dd2f0128518` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.014057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e021938485423d8c310dd2f0128518)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.015936 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.059175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e021938485423d8c310dd2f0128518'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.059878 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e021938485423d8c310dd2f0128518'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.066814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e021938485423d8c310dd2f0128518'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.073522 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e021938485423d8c310dd2f0128518'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.074117 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e021938485423d8c310dd2f0128518'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.091579 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1447/2879] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:37:00 2022] DELETE /identity/v3/users/a5e021938485423d8c310dd2f0128518 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:01.096135 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.096379 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.096694 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.119295 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:01.121422 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:01.183667 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1433/2880] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:01.193515 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.194913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:01.251071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58f325-2c18-43f2-9636-9c6b70de3f3a admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.254636 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836aa249-8f76-4a1b-a2d6-27bf1ac97602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.255276 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aa249-8f76-4a1b-a2d6-27bf1ac97602 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.255534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aa249-8f76-4a1b-a2d6-27bf1ac97602 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.255803 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836aa249-8f76-4a1b-a2d6-27bf1ac97602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.265055 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aa249-8f76-4a1b-a2d6-27bf1ac97602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.266497 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836aa249-8f76-4a1b-a2d6-27bf1ac97602 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.275537 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1448/2881] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:01.532177 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.533648 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:01.547351 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f2262-9c4b-4f53-988a-b201f9225fe9 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.552007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.552897 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.553180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.553455 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] PATH_INFO: `/v3/projects/97c79be6f62540e09e602c73c6f4ecb5` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.554647 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c79be6f62540e09e602c73c6f4ecb5)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.556624 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.604999 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c79be6f62540e09e602c73c6f4ecb5', 'request_id': 'req-730379c3-bba1-4d14-bf60-0897d6920d42'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.610674 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c79be6f62540e09e602c73c6f4ecb5', 'request_id': 'req-730379c3-bba1-4d14-bf60-0897d6920d42'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.616585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c79be6f62540e09e602c73c6f4ecb5', 'request_id': 'req-730379c3-bba1-4d14-bf60-0897d6920d42'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.628771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] The token cache is being invalidate because project 97c79be6f62540e09e602c73c6f4ecb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:37:01.629051 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.629686 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-730379c3-bba1-4d14-bf60-0897d6920d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:01.631212 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1434/2882] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:37:01 2022] DELETE /identity/v3/projects/97c79be6f62540e09e602c73c6f4ecb5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:01.636188 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.636674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.636815 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.653856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:01.655024 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:01.711486 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1449/2883] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:01.719144 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.719430 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.719707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.738695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:01.739859 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:01.748061 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1435/2884] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:01.752050 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.761279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdb49d-c5d3-4e83-a60b-d8071f2f0247 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.766287 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.766923 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.767201 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.767475 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.768181 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:37:01.768481 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.770076 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.781085 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1450/2885] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:37:01.785312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.795057 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ff608-9b85-4c94-96a7-3d7d81affdfc None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.799865 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.800494 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.800807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.801079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.801747 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.803604 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.828003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1436/2886] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:37:01 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:01.832143 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.842294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462d70-4ac0-409b-a99f-c51ed6dec2b1 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.896016 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.896604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.896914 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.897189 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.897861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.899759 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.919593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1451/2887] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:37:01 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:01.923592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.933441 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecfe04-9d04-4b1f-998b-35f17e807d4c admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.938586 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.939231 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.939510 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.939799 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.940495 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.942095 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.952050 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1437/2888] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:01.955875 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.966478 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9be15e-7958-43cf-8921-d35c89eafc47 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:01.971138 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fda78-b645-4f76-a3e0-3b177899bafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:01.971792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fda78-b645-4f76-a3e0-3b177899bafe admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:01.972074 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fda78-b645-4f76-a3e0-3b177899bafe admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:01.972358 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fda78-b645-4f76-a3e0-3b177899bafe admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:01.973113 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fda78-b645-4f76-a3e0-3b177899bafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:01.974687 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fda78-b645-4f76-a3e0-3b177899bafe admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:01.987954 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1452/2889] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:01.989012 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:01.999122 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f6c7b-4dd4-4ee7-a99c-d95640371228 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:02.003611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:02.004462 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:02.004766 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:02.005036 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] PATH_INFO: `/v3/projects/6aaf688709e846bd936da4c204cf646b/users/d0561a9cbf1941d887de07be56e39a14/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:02.018738 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaf688709e846bd936da4c204cf646b, user_id=d0561a9cbf1941d887de07be56e39a14, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:02.021446 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:02.034934 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1438/2890] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:37:01 2022] PUT /identity/v3/projects/6aaf688709e846bd936da4c204cf646b/users/d0561a9cbf1941d887de07be56e39a14/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:02.045930 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:02.046215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:02.046489 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:02.072374 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] MFA Rules not processed for user `d0561a9cbf1941d887de07be56e39a14`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:02.073647 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:02.133965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1453/2891] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:37:02.145015 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:02.146771 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:02.201978 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c6f6d-a048-4e9e-92c6-8f93969b4bb0 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:02.205695 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dd9d9-b69e-4e9d-bf36-b817f5551292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:02.206335 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dd9d9-b69e-4e9d-bf36-b817f5551292 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:02.206616 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dd9d9-b69e-4e9d-bf36-b817f5551292 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:02.206891 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dd9d9-b69e-4e9d-bf36-b817f5551292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:02.215211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dd9d9-b69e-4e9d-bf36-b817f5551292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:02.217168 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414dd9d9-b69e-4e9d-bf36-b817f5551292 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:02.225758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1439/2892] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:04.738861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:04.798609 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f3761-cab0-4566-8053-20ce7798a160 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:04.802980 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:04.803824 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:04.804101 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:04.804370 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] PATH_INFO: `/v3/users/d0561a9cbf1941d887de07be56e39a14` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:04.805547 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0561a9cbf1941d887de07be56e39a14)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:04.807427 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:04.850324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0561a9cbf1941d887de07be56e39a14'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:04.850965 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0561a9cbf1941d887de07be56e39a14'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:04.858057 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0561a9cbf1941d887de07be56e39a14'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:04.864445 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0561a9cbf1941d887de07be56e39a14'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:04.865032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0561a9cbf1941d887de07be56e39a14'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:04.881879 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1454/2893] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:37:04 2022] DELETE /identity/v3/users/d0561a9cbf1941d887de07be56e39a14 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:04.886311 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:04.886790 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:04.886903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:04.903594 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:04.904808 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:04.961343 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1440/2894] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:04.971511 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:04.972891 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:05.027388 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20c88f-c2f8-475c-8790-f0859566590b admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.031552 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37eb418-59db-4cad-baf6-514b7b9cf46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.032184 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37eb418-59db-4cad-baf6-514b7b9cf46b service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.032466 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37eb418-59db-4cad-baf6-514b7b9cf46b service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.032774 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37eb418-59db-4cad-baf6-514b7b9cf46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.042011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37eb418-59db-4cad-baf6-514b7b9cf46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.043698 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37eb418-59db-4cad-baf6-514b7b9cf46b service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.053532 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1455/2895] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:05.313782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.315206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:05.327707 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53abad-461c-4649-b993-43122a9826b8 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.332307 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.333198 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.333481 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.333752 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] PATH_INFO: `/v3/projects/6aaf688709e846bd936da4c204cf646b` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.334941 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aaf688709e846bd936da4c204cf646b)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.336924 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.383270 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aaf688709e846bd936da4c204cf646b', 'request_id': 'req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:05.388975 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aaf688709e846bd936da4c204cf646b', 'request_id': 'req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:05.394923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aaf688709e846bd936da4c204cf646b', 'request_id': 'req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:05.406866 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] The token cache is being invalidate because project 6aaf688709e846bd936da4c204cf646b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:37:05.407150 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:05.407758 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-dd5035b4-6ed2-4cbd-b6f1-6a095d0a5e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:05.409303 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1441/2896] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:37:05 2022] DELETE /identity/v3/projects/6aaf688709e846bd936da4c204cf646b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:37:05.414064 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.414348 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.414619 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.432838 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:05.434005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:05.489607 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1456/2897] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 16 19:37:05.498448 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.498733 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.499067 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.517823 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:05.519004 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:05.527134 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1442/2898] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:05.531679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.540860 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec256-3209-48bc-b5b2-833f561ec8c2 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.545593 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.546226 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.546496 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.546766 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.547472 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:37:05.547768 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.549392 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.560437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1457/2899] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 16 19:37:05.564451 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.573257 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985e323-db5e-4dd3-84ae-b3a5ac28e2f6 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.578300 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.578923 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.579200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.579471 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.580180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.582079 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.606236 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1443/2900] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:37:05 2022] POST /identity/v3/projects => generated 365 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:05.610194 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.619841 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577230c-5a8a-4ffd-a082-fc748f29d871 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.673023 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.673667 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.673941 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.674207 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.674867 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.677147 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.694985 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1458/2901] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:37:05 2022] POST /identity/v3/users => generated 283 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 16 19:37:05.698960 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.709626 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadedb5-6042-425c-bedb-7ffdc2249743 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.714140 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.714786 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.715066 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.715338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.716074 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.718251 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.728258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1444/2902] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:05.731784 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.741270 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb1e5e-9a0b-4246-9882-75af9f892ed2 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.745895 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca22756-191a-4734-93bb-676a0d933231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.746534 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca22756-191a-4734-93bb-676a0d933231 admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.746811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca22756-191a-4734-93bb-676a0d933231 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.747086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca22756-191a-4734-93bb-676a0d933231 admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.748300 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca22756-191a-4734-93bb-676a0d933231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.749943 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca22756-191a-4734-93bb-676a0d933231 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.759744 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1459/2903] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:05.764483 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.779641 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9133b-44fb-4037-88bb-ee50ccd6e03f admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.786384 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.787761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.788190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.788611 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] PATH_INFO: `/v3/projects/90f98be7e030435193c3edc41e9dfd12/users/3a8697dcd4674ff4b74e80f621519a0a/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.807657 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f98be7e030435193c3edc41e9dfd12, user_id=3a8697dcd4674ff4b74e80f621519a0a, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:05.812499 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:05.825340 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1445/2904] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:37:05 2022] PUT /identity/v3/projects/90f98be7e030435193c3edc41e9dfd12/users/3a8697dcd4674ff4b74e80f621519a0a/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:05.836195 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.836487 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.836786 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:05.862011 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] MFA Rules not processed for user `3a8697dcd4674ff4b74e80f621519a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:05.863215 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:05.924379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1460/2905] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:05.935907 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:05.938016 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:05.991927 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a5005-6956-430f-8e26-228d5e9387ac admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:05.995509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e60b49-6724-4239-9573-f60f03c9135e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:05.996139 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e60b49-6724-4239-9573-f60f03c9135e service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:05.996417 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e60b49-6724-4239-9573-f60f03c9135e service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:05.996717 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e60b49-6724-4239-9573-f60f03c9135e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:06.005154 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e60b49-6724-4239-9573-f60f03c9135e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:06.006563 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e60b49-6724-4239-9573-f60f03c9135e service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:06.015509 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1446/2906] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:54.233395 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:54.292840 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34f9fb-4aa6-4e76-8009-4776bc9bd283 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:54.297436 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:54.298281 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:54.298556 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:54.298823 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] PATH_INFO: `/v3/users/3a8697dcd4674ff4b74e80f621519a0a` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:54.299971 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8697dcd4674ff4b74e80f621519a0a)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:54.301883 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:54.346438 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8697dcd4674ff4b74e80f621519a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.347081 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8697dcd4674ff4b74e80f621519a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.355161 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8697dcd4674ff4b74e80f621519a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.361437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8697dcd4674ff4b74e80f621519a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.362324 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8697dcd4674ff4b74e80f621519a0a'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.379516 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1461/2907] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:37:54 2022] DELETE /identity/v3/users/3a8697dcd4674ff4b74e80f621519a0a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:54.384526 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:54.385026 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:54.385121 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:54.401508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:54.402697 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:54.459165 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1447/2908] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:54.468685 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:54.470027 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:54.523842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd37d2-8789-47d5-947a-695ee71a2303 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:54.527398 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f005008-212b-4ef0-adc8-5b89acc886ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:54.528018 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f005008-212b-4ef0-adc8-5b89acc886ba service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:54.528294 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f005008-212b-4ef0-adc8-5b89acc886ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:54.528564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f005008-212b-4ef0-adc8-5b89acc886ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:54.537132 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f005008-212b-4ef0-adc8-5b89acc886ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:54.538854 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f005008-212b-4ef0-adc8-5b89acc886ba service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:54.547731 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1462/2909] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:54.805172 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:54.806627 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:54.819416 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b555-856c-42a2-a762-4c460d57e788 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:54.825035 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:54.825895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:54.826179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:54.826508 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] PATH_INFO: `/v3/projects/90f98be7e030435193c3edc41e9dfd12` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:54.827631 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f98be7e030435193c3edc41e9dfd12)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:54.834206 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:54.879807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f98be7e030435193c3edc41e9dfd12', 'request_id': 'req-413792b6-f6ab-4e8b-b343-d58ca93e8785'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.885243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f98be7e030435193c3edc41e9dfd12', 'request_id': 'req-413792b6-f6ab-4e8b-b343-d58ca93e8785'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.891175 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f98be7e030435193c3edc41e9dfd12', 'request_id': 'req-413792b6-f6ab-4e8b-b343-d58ca93e8785'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.902908 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] The token cache is being invalidate because project 90f98be7e030435193c3edc41e9dfd12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:37:54.903190 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.903787 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-413792b6-f6ab-4e8b-b343-d58ca93e8785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:37:54.905428 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1448/2910] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:37:54 2022] DELETE /identity/v3/projects/90f98be7e030435193c3edc41e9dfd12 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:54.910265 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:54.910675 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:54.910960 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:54.929856 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:54.931032 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:54.986903 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1463/2911] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:54.996199 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:54.996486 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:54.996831 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.015688 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:55.016922 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:55.024942 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1449/2912] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:55.029172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.038439 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c701f25-42d6-4a6b-8348-5edda5a7a66b None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.043425 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.044070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.044351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.044621 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] PATH_INFO: `/v3/domains` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.045622 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91542) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 16 19:37:55.045926 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.047508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.059001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1464/2913] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 16 19:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 16 19:37:55.063126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.072045 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8265137-5943-4187-9b70-80da37b3c0be None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.076592 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.077264 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.077544 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.077814 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] PATH_INFO: `/v3/projects` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.078497 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.080374 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.105008 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1450/2914] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 16 19:37:55 2022] POST /identity/v3/projects => generated 373 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:55.108908 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.118811 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecc38b-0955-4d4a-a885-81ef4d88a6bd admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.173679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.174334 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.174614 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.174890 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] PATH_INFO: `/v3/users` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.175564 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.177502 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.200175 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1465/2915] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 16 19:37:55 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 16 19:37:55.201516 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.212126 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9dd8b-16e6-452d-8f07-b05034d53087 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.216748 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.217394 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.217675 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.217948 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] PATH_INFO: `/v3/roles` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.218666 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.220240 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.230970 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1451/2916] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 16 19:37:55.234842 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.244655 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976948c-db65-44d5-8d17-8dd3f0cc62eb admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.249154 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a63d7-37a5-4eea-9c52-0e5894625a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.249792 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a63d7-37a5-4eea-9c52-0e5894625a1d admin admin] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.250070 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a63d7-37a5-4eea-9c52-0e5894625a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.250344 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a63d7-37a5-4eea-9c52-0e5894625a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.251052 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a63d7-37a5-4eea-9c52-0e5894625a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.252610 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a63d7-37a5-4eea-9c52-0e5894625a1d admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.263982 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1466/2917] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 16 19:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 16 19:37:55.267793 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.277905 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d7c45-6e90-4cfb-995c-7554a254e090 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.282580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.283436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.283712 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.283980 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] PATH_INFO: `/v3/projects/662502234adc4b618a03137e850c9653/users/bdd3a235ad554297848d0ef96765783f/roles/c1095be460574bfbb5445ac9e464a9b7` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.298685 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=662502234adc4b618a03137e850c9653, user_id=bdd3a235ad554297848d0ef96765783f, role_id=c1095be460574bfbb5445ac9e464a9b7)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.301358 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.314826 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1452/2918] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 16 19:37:55 2022] PUT /identity/v3/projects/662502234adc4b618a03137e850c9653/users/bdd3a235ad554297848d0ef96765783f/roles/c1095be460574bfbb5445ac9e464a9b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:37:55.324916 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6001175-6c20-44d0-a43b-2e5cf1919bbc None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.325203 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6001175-6c20-44d0-a43b-2e5cf1919bbc None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.325505 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6001175-6c20-44d0-a43b-2e5cf1919bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.351790 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-d6001175-6c20-44d0-a43b-2e5cf1919bbc None None] MFA Rules not processed for user `bdd3a235ad554297848d0ef96765783f`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:55.353065 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d6001175-6c20-44d0-a43b-2e5cf1919bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:55.413414 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1467/2919] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:55.424415 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:55.426093 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:55.479843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd52a0-c076-496c-b026-d6a12edeb4e3 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:55.483477 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfc10bc677cc4cf4b673fdcbd937d352', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:55.484105 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:55.484386 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:55.484684 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:55.492743 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:55.494163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:55.502724 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1453/2920] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:56.822642 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:56.823120 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:56.823205 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:56.839760 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] MFA Rules not processed for user `bdd3a235ad554297848d0ef96765783f`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:37:56.840958 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:56.847865 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1468/2921] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:37:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:37:57.300412 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:57.301856 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:57.305430 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:37:57.306788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:57.362925 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddc079-092c-4041-ab1c-a2afa2749cf5 None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:57.366492 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:57.367664 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:57.367947 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:57.368219 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:57.370005 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:57.374630 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec130baa-09fd-4fe9-80c5-4a0599319575 service cinder] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:37:57.379524 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:37:57.380156 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:37:57.380436 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:37:57.380761 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:37:57.382580 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:37:57.393725 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:57.395288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:57.405548 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1454/2922] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:37:57.438384 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:37:57.439814 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfcdcd-496a-4e81-80b4-5d0d6f246a2f service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:37:57.450901 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1469/2923] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:37:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:38:06.613889 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:06.615299 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:06.631863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd724304-5f40-4d87-a741-683f54d6bad1 service placement] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:06.637484 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:06.638114 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:06.638389 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:06.638659 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:06.640959 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:06.654022 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:06.655599 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:06.669312 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1455/2924] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:38:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:38:12.836942 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965dd65-5fe2-4415-9b98-05185faa19f6 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:12.837413 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965dd65-5fe2-4415-9b98-05185faa19f6 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:12.837513 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965dd65-5fe2-4415-9b98-05185faa19f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:12.854674 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-b965dd65-5fe2-4415-9b98-05185faa19f6 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:38:12.855876 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-b965dd65-5fe2-4415-9b98-05185faa19f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:12.915984 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1470/2925] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:38:12.961669 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:12.963260 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:13.029181 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137ee7-6c70-48ff-b4aa-5dbe338ee833 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:13.034268 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:13.034903 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:13.035183 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:13.035457 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:13.043784 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:13.045400 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:13.053601 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1456/2926] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:38:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:38:13.543298 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db960f61-f8c2-4cf3-ae05-c14509ec86a2 None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:13.543634 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db960f61-f8c2-4cf3-ae05-c14509ec86a2 None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:13.543934 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db960f61-f8c2-4cf3-ae05-c14509ec86a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:13.562932 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-db960f61-f8c2-4cf3-ae05-c14509ec86a2 None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:38:13.564103 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-db960f61-f8c2-4cf3-ae05-c14509ec86a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:13.571170 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1471/2927] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 16 19:38:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:38:13.583321 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:13.593581 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4950693-deaf-4127-97ae-0ea703d2d666 service swift] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:13.598540 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:13.599163 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:13.599440 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:13.599719 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:13.607926 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:13.609523 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:13.617255 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1457/2928] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:38:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:38:13.798827 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28e274-5b84-4623-9b4f-221b885fa33d None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:13.799110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28e274-5b84-4623-9b4f-221b885fa33d None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:13.799382 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28e274-5b84-4623-9b4f-221b885fa33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:13.816944 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-9d28e274-5b84-4623-9b4f-221b885fa33d None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:38:13.818100 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-9d28e274-5b84-4623-9b4f-221b885fa33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:13.825030 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1472/2929] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 16 19:38:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:38:13.837244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:13.847460 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e06134-9448-4b95-93c7-15b1b0f2b3b3 service swift] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:13.853109 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:13.853730 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:13.854007 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:13.854278 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:13.862468 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:13.864031 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:13.871906 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1458/2930] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:38:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:38:14.174900 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] REQUEST_METHOD: `POST` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:14.175389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:14.175488 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:14.192533 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.auth.core [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] MFA Rules not processed for user `7bab459b1a3e48b0973482e9193aeb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=91542) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:38:14.193742 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:14.201172 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1473/2931] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 16 19:38:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:38:14.214125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:14.224287 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242135c2-63e4-4c7f-b931-549132455b9f service swift] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:14.229162 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7deb0-4e23-48b2-b31d-c36333b879ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16b2497bffb3487bbd025f22463d5a79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:14.229777 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7deb0-4e23-48b2-b31d-c36333b879ea service swift] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:14.230055 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7deb0-4e23-48b2-b31d-c36333b879ea service swift] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:14.230354 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7deb0-4e23-48b2-b31d-c36333b879ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:14.238211 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7deb0-4e23-48b2-b31d-c36333b879ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:14.239795 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7deb0-4e23-48b2-b31d-c36333b879ea service swift] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:14.247652 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1459/2932] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 16 19:38:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 16 19:38:16.243123 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:16.302259 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea601fc-653e-4d05-89da-72bb4c73c71c None None] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:16.307462 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:16.308355 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:16.308601 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:16.308904 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] PATH_INFO: `/v3/users/bdd3a235ad554297848d0ef96765783f` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:16.310087 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd3a235ad554297848d0ef96765783f)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:16.312019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:16.357158 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd3a235ad554297848d0ef96765783f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.357804 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd3a235ad554297848d0ef96765783f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.364788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd3a235ad554297848d0ef96765783f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.373271 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd3a235ad554297848d0ef96765783f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.373871 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd3a235ad554297848d0ef96765783f'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.390592 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1474/2933] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:38:16 2022] DELETE /identity/v3/users/bdd3a235ad554297848d0ef96765783f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:38:16.395112 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] REQUEST_METHOD: `POST` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:16.395585 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:16.395692 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:16.412575 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.auth.core [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] MFA Rules not processed for user `2ef3d63903f749da901176bdd82045cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91543) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 16 19:38:16.413782 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:16.469243 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1460/2934] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 16 19:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 16 19:38:16.479369 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:16.480751 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:16.543210 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243061dd-e2bf-43ac-835f-c4363f892ef8 admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:16.547343 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:16.547969 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:16.548249 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:16.548545 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:16.557708 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:16.559110 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:16.568483 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1475/2935] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 16 19:38:16.728244 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:16.730180 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:16.746527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ac57c-0ba3-4d35-81d7-844fd6e2b223 None None] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:16.751493 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:16.752476 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:16.752683 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:16.753002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] PATH_INFO: `/v3/projects/662502234adc4b618a03137e850c9653` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:16.754200 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=662502234adc4b618a03137e850c9653)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:16.756087 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:16.800807 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662502234adc4b618a03137e850c9653', 'request_id': 'req-259a3245-089f-4aa9-b459-fc27304317cb'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.806677 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662502234adc4b618a03137e850c9653', 'request_id': 'req-259a3245-089f-4aa9-b459-fc27304317cb'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.813238 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662502234adc4b618a03137e850c9653', 'request_id': 'req-259a3245-089f-4aa9-b459-fc27304317cb'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.825989 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] The token cache is being invalidate because project 662502234adc4b618a03137e850c9653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91543) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:38:16.826382 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.827002 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:16.828535 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1461/2936] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:38:16 2022] DELETE /identity/v3/projects/662502234adc4b618a03137e850c9653 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 16 19:38:22.968810 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:22.977454 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9dec9a-332b-4087-b656-433826196462 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:23.024086 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:23.024788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:23.025071 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:23.025346 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:23.027372 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:23.082604 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:23.084019 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:23.092574 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1476/2937] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:38:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:38:23.918530 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:23.920075 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:24.005458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a3245-089f-4aa9-b459-fc27304317cb admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:24.013288 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b163f5034b7a4d96a224c07b9c1428c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:24.013877 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:24.014160 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:24.014433 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:24.016239 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:24.035895 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:24.037881 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:24.050527 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1462/2938] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:38:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:38:27.766576 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:27.768001 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:27.779351 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9241fe-a230-4014-b5bc-220a28996415 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:27.782948 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e653231605f345b0846ac42508e7d9a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:27.783577 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] REQUEST_METHOD: `GET` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:27.783861 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:27.784137 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:27.785858 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:27.846094 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:27.847437 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:27.856869 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1477/2939] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 16 19:38:33.961863 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:33.964532 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:34.010046 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7296974-3e21-4092-8ad8-cd1d9b4e91b8 service nova] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:34.015319 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:34.016179 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:34.016458 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:34.016765 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] PATH_INFO: `/v3/users/17e016e1e171494c9932df04761fb738` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:34.017928 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e016e1e171494c9932df04761fb738)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:34.019847 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:34.063184 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e016e1e171494c9932df04761fb738'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.069693 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e016e1e171494c9932df04761fb738'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.070338 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e016e1e171494c9932df04761fb738'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.077704 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e016e1e171494c9932df04761fb738'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.078313 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e016e1e171494c9932df04761fb738'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.095475 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1463/2940] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:38:33 2022] DELETE /identity/v3/users/17e016e1e171494c9932df04761fb738 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:38:34.261978 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:34.295459 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94146e9-8f98-447d-ac37-f1f61f9d0401 service neutron] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:34.348625 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:34.349508 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:34.349788 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:34.350062 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] PATH_INFO: `/v3/projects/1954e5ac9597438b97fd388aa735a467` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:34.351229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1954e5ac9597438b97fd388aa735a467)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:34.353229 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:34.397284 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1954e5ac9597438b97fd388aa735a467', 'request_id': 'req-7b5aca4b-4261-41b9-b436-60bd76aaa04e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.402913 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1954e5ac9597438b97fd388aa735a467', 'request_id': 'req-7b5aca4b-4261-41b9-b436-60bd76aaa04e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.410717 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1954e5ac9597438b97fd388aa735a467', 'request_id': 'req-7b5aca4b-4261-41b9-b436-60bd76aaa04e'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.421843 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] The token cache is being invalidate because project 1954e5ac9597438b97fd388aa735a467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:38:34.422389 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.423009 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.424521 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1478/2941] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:38:34 2022] DELETE /identity/v3/projects/1954e5ac9597438b97fd388aa735a467 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:38:34.428487 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:34.429901 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:34.446258 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5761114-d285-44f1-9d32-1602de4636b1 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:34.500843 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:34.501721 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:34.502003 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:34.502271 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] PATH_INFO: `/v3/users/cbaa15f61af6467c974e66562fc6f0b1` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:34.503439 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbaa15f61af6467c974e66562fc6f0b1)` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:34.505383 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:34.547625 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbaa15f61af6467c974e66562fc6f0b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.554201 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbaa15f61af6467c974e66562fc6f0b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.554850 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbaa15f61af6467c974e66562fc6f0b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.561512 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbaa15f61af6467c974e66562fc6f0b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.562125 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.notifications [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbaa15f61af6467c974e66562fc6f0b1'} {{(pid=91543) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.578608 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1464/2942] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 16 19:38:34 2022] DELETE /identity/v3/users/cbaa15f61af6467c974e66562fc6f0b1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:38:34.745679 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Authenticating user token {{(pid=91542) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:38:34.747108 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.fernet_utils [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91542) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:38:34.760118 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aca4b-4261-41b9-b436-60bd76aaa04e admin admin] Validating token access rules against request {{(pid=91542) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:38:34.820989 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ef3d63903f749da901176bdd82045cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b27a519f15bc4f1e98fc811594ab8cdd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91542) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:38:34.821834 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:38:34.822111 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] SCRIPT_NAME: `/identity` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:38:34.822379 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] PATH_INFO: `/v3/projects/2f6dcabb291745e5820de12dfe8b92f9` {{(pid=91542) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:38:34.823561 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6dcabb291745e5820de12dfe8b92f9)` {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:38:34.825469 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] RBAC: Authorization granted {{(pid=91542) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:38:34.870236 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6dcabb291745e5820de12dfe8b92f9', 'request_id': 'req-861033ff-689e-41be-99b0-13f00f7ce093'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.875738 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6dcabb291745e5820de12dfe8b92f9', 'request_id': 'req-861033ff-689e-41be-99b0-13f00f7ce093'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.882359 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6dcabb291745e5820de12dfe8b92f9', 'request_id': 'req-861033ff-689e-41be-99b0-13f00f7ce093'} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.892880 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] The token cache is being invalidate because project 2f6dcabb291745e5820de12dfe8b92f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91542) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 16 19:38:34.893159 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.893770 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: DEBUG keystone.notifications [None req-861033ff-689e-41be-99b0-13f00f7ce093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91542) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 16 19:38:34.895279 dv-cinder-841125-5-iscsi devstack@keystone.service[91542]: [pid: 91542|app: 0|req: 1479/2943] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 16 19:38:34 2022] DELETE /identity/v3/projects/2f6dcabb291745e5820de12dfe8b92f9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 16 19:41:35.436920 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Authenticating user token {{(pid=91543) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 16 19:41:35.438332 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:41:35.508983 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065579b6-4553-48ae-9a5e-ce4225b2c8b4 admin admin] Validating token access rules against request {{(pid=91543) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 16 19:41:35.564557 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c103464c1bca4da29e61fe73b6fd1d92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25fe1818fd6341f49e3e7ccebcadfc1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91543) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 16 19:41:35.565223 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] REQUEST_METHOD: `GET` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 16 19:41:35.565505 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] SCRIPT_NAME: `/identity` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 16 19:41:35.565781 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91543) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 16 19:41:35.567519 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.fernet_utils [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91543) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 16 19:41:35.578362 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 16 19:41:35.579930 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2849ad-fa14-44e9-ba9f-9c95bc292bd0 service placement] RBAC: Authorization granted {{(pid=91543) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 16 19:41:35.589656 dv-cinder-841125-5-iscsi devstack@keystone.service[91543]: [pid: 91543|app: 0|req: 1465/2944] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 16 19:41:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)